Measurement Lab at the EU Hackathon 2011
This page contains information to help Hack4Transparency applicants get acquainted with M-Lab’s NDT and Glasnost code and data, and begin coming up with ideas and project proposals. Let us know here, if you have any questions that this page does not answer.
M-Lab is happy to be a part of the upcoming Hack4Transparency event, November 8th and 9th at the European Parliament building, in Brussels, Belgium. Hack4Transparency invites hackers in to the halls of government to help build and improve tools that further transparency and Internet quality. Chosen applicants will have their travel and accommodations paid for, and will be eligible for cash prizes. If you’re interested in participating, see application details, rules, and more information on Hack4Transparency site.
In particular, participants can contribute to one of the following areas:
- Build compelling visualizations using data collected by NDT and/or Glasnost. Feel free to use existing, open visualization tools, or create yours from scratch. For inspiration and to explore the data you’ll be working with, check out these examples of visualizations that use NDT data or DPI data. You can find below more info about the data.
- Build innovative and user-friendly multi-platform interfaces for existing M-Lab measurement tools in order to help Internet users understand the quality of their own connections. Due to their popularity, we suggest to focus on M-Lab’s NDT or Glasnost tools. However, we are also open to applications that make use of other M-Lab tools. Yuo can find below more info about NDT and Glasnost.
NDT
Where are the sources for NDT?
Where are the executable NDT clients?
Where is the documentation for NDT?
Where are the data collected by NDT?
Glasnost
Where are the sources for Glasnost?
Where are the executable Glasnost clients?
Where is the documentation for Glasnost?
Where are the data collected by Glasnost?
What is NDT?
The Network Diagnostic Tool (NDT) provides a sophisticated speed and diagnostic test. An NDT test reports more than just the upload and download speeds -- it also attempts to determine what, if any, problems limited these speeds, differentiating between computer configuration and network infrastructure problems. While the diagnostic messages are most useful for expert users, they can also help novice users by allowing them to provide detailed trouble reports to their network administrator.
Where are the sources for NDT?
The NDT source code is housed at its Google Code project. There are several sub-trees of interest within the Subversion repository hosted there. Within the trunk directory, the Applet subdirectory contains the Java code used by the current Web front end at the Measurement Labs web site. In the Android branch of the repository is the latest iteration of the mobile client for that platform.
The Google Code repository includes instructions for checking out a read only copy of the sources. Members of the Measurement Lab operations committee will be present a the Hack4Transparency event and there can discuss submitting patches from any applicants interested in contributing directly.
Where are the executable NDT clients?
It may prove useful to observe the existing clients running for inspiration and clarification of how the tool works. The current web front end can be found at the Measurement Lab web site. The underlying, older Applet version can be viewed by directly opening http://ndt.iupui.donar.measurement-lab.org:7123/. The latest beta of the Android client is available in the Android marketplace.
Where is the documentation for NDT?
The wiki at the Google Code project includes documents covering various aspects of NDT. The protocol document covers the testing life cycle and the low level details of the tests themselves. The architecture document (not yet released) will provide a high level overview of how NDT works. Much of the NDT source code has recently been updated to improve inline documentation of how the low level code itself works.
Where are the data collected by NDT?
You have different options to access the NDT data:
- Download the raw data. More info at http://measurementlab.net/data, Section 1.
- Execute SQL queries against a portion of the raw NDT dataset. More info at http://measurementlab.net/data, Section 2.
- Downlaod the aggregated statistics that are visualized at http://measurementlab.net/visualization. More info at http://measurementlab.net/visualization, Section 'Data'.
What is Glasnost?
Glasnost attempts to detect whether your Internet access provider is performing application-specific traffic shaping. Currently, you can test if your ISP is throttling or blocking email, HTTP or SSH transfer, Flash video, and P2P apps including BitTorrent, eMule and Gnutella. Glasnost is extensible, capable of running tests created by anyone with an easy to use tool called trace-emulate.
Where are the sources for Glasnost?
A public git repository of the latest Glasnost sources is available at Google Code. Instructions for cloning the sources are available at the project site.
Where are the executable Glasnost clients?
The Glasnost web client can be found at the Max Planck Institute for Software Systems project page. Currently there are options to test a variety of common internet protocols and applications for protocol specific traffic shaping. User created tests, using the trace-emulate tool, can also be executed online via a page at the MPI-SWS project site.
Where is the documentation for Glasnost?
The source code of Glasnost itself is fairly small and readable. In addition, there is a paper describing the overall design of the tool. The project site contains additional documentation on running a server and creating new tests.
Where are the data collected by Glasnost?
You have different options to access the Glasnost data:
- Download the raw data. More info at http://measurementlab.net/data, Section 1.
- Downlaod the aggregated statistics that are visualized at http://dpi.ischool.syr.edu/MLab-Data.html. More info at http://code.google.com/p/glasnost/wiki/DPIData. Please, note that if you are using the DPI data, we require that you give
- Credit to the DPI project (a reference to "the network is aware" project and link to deeppacket.info).
- Permission to display any visualizations developed using the data on deeppacket.info, with credit to the creator.
