Artificial intelligence (AI) has come a long way and the progress provides a wide range of opportunities. But a problem arises when people do not get to optimize their AI-focused projects due to lack of clear standards and benchmarks. In light of this, Google and Facebook, together with 38 other tech companies came together to found a consortium which will see to it that benchmarks are created.
In a news release, the companies said the consortium would come up with benchmarks to help standardize the machine learning. Dubbed MLPerf, the consortium includes professionals from the academia.
MLPerf will focus measuring the robustness of AI projects against the benchmarks. As such, the people behind the projects will be in a position to identify product solutions which are optimal. Ultimately, this will inspire confidence in the people behind such projects because they will be able to tell whether they are on the right track or not.
Already, the consortium introduced a machine learning standard which will be the first in the industry. Dubbed MLPerf Inference v0.5, this benchmark suite will help to measure power efficiency and system performance. This product leverages data sets and models which are carefully selected with a view on achieving perfect results in real-world applications.
Machine learning tasks in focus
Machine learning consists of various tasks but three are common. As such, MLPerf Inference v0.5 is developed around the three common tasks which include Image Classification, Object Detection and Machine Translation.
Under image classification, there will be benchmarks focused on how to best identify items in a given photo. Further, the benchmarks under object detection will help to streamline the process of isolating an object within an image from a given dataset, specifically MS-COCO. Lastly, the benchmarks for machine translation will enable products to better utilize WMT benchmark to translate from one language to the next.
According to the project’s general chair, Peter Mattson, the primary objective of the benchmarks is to sow the seed of innovation in the field of AI. Specifically, these benchmarks address the need for AI to transition from hype to reality and to attract more organizations.