Computer Vision Proof of Concept
A Story of Owning Change
About BillionGraves
As the world’s largest resource for searchable GPS cemetery data, BillionGraves endeavors to preserve precious records found in cemeteries throughout the world to assist in family history research and help people connect to their family members through past generations. Using cutting-edge technology, people around the world can capture images of headstones with their GPS locations so users worldwide can access those records anywhere.
1Strategy has been a great help and a wealth of knowledge! We’re starting to implement all of the work that has been done into a production stream now.
—Brian Moncur, Chief Technology Officer of BillionGraves
The Challenge
BillionGraves has an enthusiastic and committed user base around the world. Currently, users voluntarily take pictures of headstones, and through an app or online interface, they upload pictures with associated GPS location information. They then transcribe the headstone inscriptions (name, date of birth, date of death, relatives, etc.) into text.
However, BillionGraves recognized that the rate at which headstones could be added to the database was limited by the current workflow. They imagined a solution that could speed up the process by a factor of 10: taking handheld or drone-mounted videos and then using computer vision to identify headstones and provide images or short video clips to users for transcription.
Why Amazon Web Services
Because of prior experience with AWS for data storage and processing, BillionGraves was familiar with the potential power of leveraging the AWS ecosystem. They recognized that AWS provided a number of services with potential to drive their business to the next level, specifically Amazon S3 and Amazon SageMaker.
BillionGraves expected to ingest much larger data streams and could benefit from S3’s provisions for cost-tiered data storage, processing, web-serving and archiving. SageMaker provides a seamless, unified platform for testing, training, validating and productionizing computer vision algorithms and the post-processing necessary to achieve BillionGraves’ business goals. In addition, BillionGraves was exploring the possibility of utilizing Amazon SageMaker Ground Truth and exploring data analysis using Amazon Textract.
The Benefits
BillionGraves reached out to 1Strategy based on several previous interactions and was very familiar with 1Strategy’s extensive experience within AWS. 1Strategy worked closely with BillionGraves to develop a mixture of managed and custom services to create a pipeline for processing video data using computer vision.
Through this project with 1Strategy, machine learning enabled data acquisition from photos and videos, either from a handheld device or from drone footage. 1Strategy implemented a custom object detector in Amazon SageMaker. This single-shot detector recognized headstones and isolated them in a frame using a training sample of several thousand annotated headstone images hosted on Amazon S3, increasing the ingestion rate tenfold.
Model training involves taking a set of images of headstones and manually drawing “bounding boxes” around them using custom software developed by BillionGraves engineers. The training set consists of thousands of such labeled images that are then fed to the neural network hosted on AWS, which can learn how to replicate this process. Once the neural network is trained, new and unlabeled still images or individual video frames are shown to the model, which can identify headstones and locate them in the image.
1Strategy developed a custom algorithm in SageMaker to link the headstones in a series of adjacent frames for the purpose of finding the best frame for human readers to be able to document the annotations on the headstone. The image below is an example of the neural network at work. It has found and located three headstones in a frame from a sample video. The labels around the bounding boxes are measures of the quality of the identification and the clarity of the image.
1Strategy’s six-week rapid implementation of SageMaker’s native object detection neural network enabled BillionGraves to implement and rapidly iterate on the proof of concept and identify several data quality and processing issues almost immediately. As Brian Moncur, chief technology officer of BillionGraves put it, this input was like “rocket fuel for the project.” BillionGraves was able to avoid wasted manual effort by changing the way it collected and labeled training data. In addition, quickly analyzing the results allowed BillionGraves to understand the magnitude of the larger and more inclusive data sets they will eventually require.
“1Strategy has been a great help and a wealth of knowledge!” Moncur said. “We’re starting to implement all of the work that has been done into a production stream now.”
By working closely with BillionGraves, 1Strategy engineers were able to both collaborate and bring value to their business through education in available AWS machine learning services and techniques. 1Strategy’s hands-on approach enabled BillionGraves to create a new SageMaker pipeline for machine learning training and to implement their updated data set in an updated object detector endpoint. BillionGraves integrated the codeveloped video post-processing pipeline into their existing data pipeline and have moved into production for the end product, a stream of images for users to label, with the goal of increasing the company’s ingest rate from 250 headstones per hour to 2,400-plus headstones.
The work described in this engagement was originally completed by 1Strategy, a TEKsystems Global Services company acquired in 2019. As of June 2023, 1Strategy has fully integrated with TEKsystems Global Services to continue to deliver AWS expertise to customers. Learn more about our AWS solutions.
Discover The Power of Real Partnership
Let’s talk about the world of possibilities and how we can partner to make them a reality.