Deploying Natural Language Processing models

You can deploy a Natural Language Processing model in watsonx.ai Runtime by using Python functions or Python scripts. Both online and batch deployments are supported.

• You can deploy a Python function for online and inline batch deployments and a Python script for batch data reference deployments.
• The prediction output that is returned from <model>.run() is an object of a class specific to the concerned data model's prediction class (for example, watson_nlp.data_model.syntax.SyntaxPrediction). Such objects cannot be serialized into JSON format so the prediction output must be converted to either the Python dictionary type or JSON by using the <prediction output>.to_dict() (recommended) or <prediction output>.to_json() methods. If you don't convert the output, scoring API requests return an error. See Example of handling prediction output of an NLP model.
• You can access the location of pre-trained Watson NLP models in the Python function code by using the LOAD_PATH environment variable.
• Prediction input payload and prediction response that is returned from score() must meet the requirements that are listed in online scoring and jobs API documentation.
• Scoring requests for NLP models might fail with an Out of Memory error that is reported by the underlying JVM runtime. If an error is reported, patch the deployment to use a hardware specification with more available memory.

import watson_nlp

# Load the syntax model for English
syntax_model = watson_nlp.load('syntax_izumo_en_stock')

# Run the syntax model and print the result
syntax_prediction = syntax_model.run('Welcome to IBM!')
print(syntax_prediction)

import watson_nlp
entities_workflow = watson_nlp.load('entity-mentions_transformer-workflow_multilingual_slate.153m.distilled')
entities = entities_workflow.run('IBM\'s CEO Arvind Krishna is based in the US', language_code="en")
print(entities.get_mention_pairs())

    for input_data in input_data_row:
        targets_sentiments = targets_sentiment_model.run(input_data)
        scoring_prediction_row.append(targets_sentiments.to_dict())