Execute the following steps to export or import a table from Cloud Bigtable:
- To export a table from Cloud Bigtable, first we need to identify its column families. To do so, first install the cbt utility:
gcloud components update gcloud components install cbt
- Use the cbt tool to list out the column families and determine which column families you want to export:
cbt -instance <<INSTANCE_ID>> ls <<TABLE_NAME>>
- Create a Cloud Storage Bucket using the gsutil mb command.
- Now, the table will be exported as sequence files. To export them, download and install import/export jar for Bigtable using this URL:
curl -f -O http://repo1.maven.org/maven2/com/google/cloud/bigtable/bigtable-beam-import/1.1.2/bigtable-beam-import-1.1.2-shaded.jar
- Run the following command and replace <<Export_path>> and <<Temp_Path>> as per your requirement:
java -jar bigtable-beam-import-1.1.2-shaded.jar export --runner=dataflow --project=<<Project_ID>> --bigtableInstanceId=<<INSTANCE_ID>> --bigtableTableId=<<TABLE_ID>> --destinationPath=gs://<<BUCKET_NAME>>/<<EXPORT_PATH>> --tempLocation=gs://<<BUCKET_NAME>>/<<TEMP_PATH>> --maxNumWorkers=<<10x_NUMBER_OF_NODES>>
- Similarly, to import a table into Cloud Bigtable, use the following command:
java -jar bigtable-beam-import-1.1.2-shaded.jar import --runner=dataflow --project=<<PROJECT_ID>> --bigtableInstanceId=<<INSTANCE_ID>> --bigtableTableId=<<TABLE_ID>> --sourcePattern='gs://<<BUCKET_NAME>>/<<EXPORT_PATH>>/part-*' --tempLocation=gs://<<BUCKET_NAME>>/<<TEMP_PATH>> --maxNumWorkers=<<5x_NUMBER_OF_NODES>>