Oracle has developed an open source specification for transmitting tensor data, which the company wants to become a standard for machine learning.
Called GraphPipe, the specification provides a protocol for network data transmission. GraphPipe is intended to bring the efficiency of a binary, memory-mapped format while being simple and light on dependencies. There also are clients and servers for deploying and querying machine learning models from any framework.
It includes:
- A set of flatbuffer definitions. Flatbuffers are similar to Google protocol buffers, with an additional benefit of avoiding memory copy during deserialization. Flatbuffer definitions provide a request message that includes input, tensors, input names, and output names.
- Guidelines for serving models.
- Examples of serving models from various machine learning frameworks.
- Client libraries for querying models served through GraphPipe. Clients are available for Python, Go, and Java. There’s a plugin for Google’s TensorFlow library, for including a remote model inside a local TensorFlow graph.
With GraphPipe, a remote model accepts a request message and returns one tensor per output name. The model also provides metadata about types and shapes of inputs and outputs.
Oracle says GraphPipe addresses three persistent challenges in machine learning:
- There is no standard for model serving APIs, leaving developers likely stuck with whatever the framework provides, which could be protocol buffers or custom JSON. An application will generally need a custom client to talk to the deployed model. The situation worsens if multiple frameworks are being used.
- Building a model server can be complicated.
- Serving tensor data from a complex model via a Python-JSON API is insufficient for performance-critical applications.
Where to download GraphPipe
You can download GraphPipe from Oracle’s GitHub repo site.
