Big Data is concerned with ways to systematically analyze and extract value from data sets that are too large or too complex to be dealt with by traditional processing methods.

Sources and Owners of the Data

The data may come from our clients or other private sources.  Much of the data is drawn from public sources and is free for use by all.  Finally, some of this data belongs to Olameter.

Forms of Data

  • Work orders: meter installs, underground locates and other site service tickets
  • Pictures: street view and aerial photos
  • Sketches: dimensioned drawings, maps, schematics, flowcharts
  • Sensor data: include meters, line sensors, barcode scans, inertial readings, compass headings
  • Vendor’s technical specifications 
  • Weather data: temperature, pressure, humidity, luminosity, wind direction and strength

Architecture

Olameter’s Spatial Data Infrastructure will make use of Kubernetes (k8s) architecture ­– a portable, extensible open-source platform for managing containerized workloads and services, in combination with Minio, a private distributed object storage system.   It will enable Olameter:

  • To handle large quantities of data that, in addition, comes in a wide variety of formats
  • To extract only those data elements which are both necessary and sufficient for the specific application, whether initiated by an AI agent or a technician in the field
  • To manage redundant data files both for reasons of speed of access, as well as for security
  • To migrate easily to such cloud service platforms as AWS, Google, Microsoft Azure, etc.

Use of the Data

The data originates with, and is used by, many applications, whether run from Olameter’s Network Operations Center or from the field.  As new data arrives, machine-learning agents are constantly reworking all the data – applying new algorithms, recognizing new objects, re-computing co-ordinates and developing new inferences.

Security

Security is a key reason why Olameter hosts its own object storage servers.  Minio provides support for client and server-side encryption of data, using secure ciphers including AES-256-GCM, ChaCha20-Poly1305, and AES-CBC. Encrypted objects are tamper-proofed with authenticated encryption with associated data (AEAD) server-side encryption which assures the confidentiality and authenticity of data.