Managing a Million Data Points: A How-To Guide

$17.50 plus tax (Refund Policy)

Buy Article:


The Hampton Roads Sanitation District (HRSD), serving the Tidewater Area of Virginia, has been tasked to maintain strict data reliability of its flow, pressure, and rain gauge monitoring sites. Per the Environmental Protection Agency (EPA) Administrative Order (AO), HRSD is required to maintain 75% data reliability for each individual meter during each monthly monitoring period and 90% data reliability for all data for each type of monitoring sensor or meter during qualifying wet weather events. HRSD has deployed 122 pressure monitoring sites, 175 flow monitoring sites and 66 rain gauge sites. Approximately 87,000 data points must go through a daily QA/QC process to ensure data validity and reliability for the flow meters and pressure sensors alone. The automation of data quality alerts through SQL statements has created a streamlined, efficient method to handle thousands of data points on a daily basis to identify system changes, system reaction, and instrument fouling.

Keywords: Data Analysis; SQL; Telog; wastewater

Document Type: Research Article


Publication date: January 1, 2010

More about this publication?
  • Proceedings of the Water Environment Federation is an archive of papers published in the proceedings of the annual Water Environment Federation® Technical Exhibition and Conference (WEFTEC® ) and specialty conferences held since the year 2000. These proceedings are not peer reviewed.

    WEF Members: Sign in (right panel) with your IngentaConnect user name and password to receive complimentary access.
  • Subscribe to this Title
  • Membership Information
  • About WEF Proceedings
  • WEFTEC Conference Information
  • ingentaconnect is not responsible for the content or availability of external websites



Share Content

Access Key

Free Content
Free content
New Content
New content
Open Access Content
Open access content
Subscribed Content
Subscribed content
Free Trial Content
Free trial content
Cookie Policy
Cookie Policy
ingentaconnect website makes use of cookies so as to keep track of data that you have filled in. I am Happy with this Find out more