This feature is available to Algorithmia Enterprise users only.
Algorithmia Insights is an algorithm metrics pipeline providing you access to your algorithm metrics payload for each algorithm session so you can measure, monitor, and manage your algorithms in production.
With Algorithmia Insights you’ll be able to opt in to exporting your algorithm’s operational metrics to external monitoring and alerting tools. This means you’ll be able to monitor your algorithm’s execution time as well as capture any model inference metrics you expose from your algorithm’s output such as predictions or accuracy. Any metrics you opt to collect will be captured and exported in a payload and, with help from a Platform Administrator, can be connected to your external systems for monitoring and alerting.
Types of Algorithmia Insights metrics
Algorithmia Insights metrics fall into two categories:
- Operational metrics
- Inference-related metrics
Operational metrics include (but are not limited to) execution time, request ID, algorithm owner, algorithm name, algorithm version, timestamp, etc. When Algorithmia Insights is turned on for a particular algorithm version, these are automatically collected and reported without any additional action necessary on the part of the algorithm author or manager.
Inference-related metrics are defined by the algorithm author in algorithm code. Each metric is defined by a title, like
cute_cats_detected, and a numeric value, like
24. Multiple inference-related metrics can be reported at one time, and the set of metrics reported does not need to be the same from one algorithm execution to another.
Setting up Algorithmia Insights on your cluster
The Kafka connection required for Insights is a global configuration on your Algorithmia cluster. If Insights isn’t already available on your cluster, ask your cluster admin to configure a connection to Kafka.
Using Algorithmia Insights in an algorithm
Each algorithm in your Algorithmia cluster has independent settings for whether to collect and report Algorithmia Insights. Additionally, each published version of your algorithm can have Algorithmia Insights turned on or off.
Creating inference-related metrics for your algorithm
Algorithmia client libraries are used to report inference-related metrics that you create inside of your algorithm code. Please see the “Publishing Algorithmia Insights” section of the appropriate client library guide for the language that you’re using.
You can report up to 25 metrics in your code. The Algorithmia Platform will report additional metrics that do not count towards these 25.
Additionally, the total size of your metrics data (keys and values) must not exceed 25 kilobytes.
Finally, there are several reserved metric names that must not be used by client code, as they will conflict with automatically created metrics for each algorithm execution:
Enabling Algorithmia Insights for an algorithm version
Enabling Algorithmia Insights for an algorithm version happens during algorithm publishing.
When using the web UI publishing workflow, a toggle will appear to enable Algorithmia Insights for the specific algorithm version that you are publishing.
When using the HTTP API or client libraries to publish your algorithm, passing the key value pair
"insights_enabled":true in the
settings hash will enable Algorithmia Insights for your algorithm. For example, when using cURL:
Once you’ve published your algorithm with Algorithmia Insights enabled, all executions of that specific version of the algorithm will trigger your Algorithmia Insights to be collected and reported to the Kafka broker that your Algorithmia Platform Administrator has set up.
Viewing Algorithmia Insights status for algorithm versions
The “Versions” tab of the algorithm page shows which published versions of that algorithm have Algorithmia Insights enabled:
If you’re new to Algorithmia and would like to learn more about our product and the Algorithmia Insights metrics pipeline, please contact our sales team. We’d love to hear from you!