The ibm-watsonx-gov Python SDK is a Python library that you can use to programatically monitor, manage, and govern machine learning models and generative AI assets. You can use the Python SDK to calculate metrics and algorithms in
a notebook runtime environment or offloaded as Spark jobs against IBM Analytics Engine for model evaluations.
Use the ibm-watsonx-govPython SDK, to calculate evaluation metrics and generate insights. You can automate these tasks by using modules and
integrating them with your application. You can also use sample notebooks to compute metrics.
Modules
Copy link to section
The Python SDK supports the following modules that can help you automate tasks for model evaluations and generate insights:
The Python SDK supports metrics that help you evaluate traditional machine learning model evaluations and prompt template evaluations for generative AI assets. For more information, see Evaluation metrics.
The following metrics are currently available only with the Python SDK:
Detects the English-language spelling errors in the model input questions
The following metric category is also available only with the Python SDK:
Content validation metrics
Copy link to section
Content validation metrics use string-based functions to analyze and validate generated LLM output text. The input must contain a list of generated text from your LLM to generate content validation metrics.
If the input does not contain transaction records, the metrics measure the ratio of successful content validations and compares the ratio to the total number of validations. If the input contains transaction records, the metrics measure the
ratio of successful content validations when compared to the total number of validations and calculate validation results with the specified record_id.
You can calculate the following content validation metrics:
About cookies on this siteOur websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising.For more information, please review your cookie preferences options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.