Q: How do I access M-Lab’s raw NDT7 data buckets? I get permission errors with gcloud.
A: Read access to the M-Lab data buckets is granted to Google accounts that are members of the M-Lab Discuss group. Anonymous requests are denied. To access the data:
- Subscribe the Google account you want to use to the M-Lab Discuss group.
- Log in to gcloud with that same account:
gcloud auth login- Access the data by specifying the bucket path:
gcloud storage ls gs://archive-measurement-lab/ndt/ndt7/Listing all buckets in the project fails even for members, because it is a restricted operation. Always access the data directly by specifying the bucket path, as shown above.
For non-interactive access with a service account, see Archival Data.
Data Organization
Raw NDT7 data is organized by date under the path:
gs://archive-measurement-lab/ndt/ndt7/YYYY/MM/DD/For example, to access data from a specific date:
gcloud storage ls gs://archive-measurement-lab/ndt/ndt7/2024/04/05/If you get a 401 or 403 error, check that gcloud config get-value account shows the account you subscribed to the Discuss group.