fix: Added explanations output to forecasting notebook (#613)

* Added explanations output to forecasting notebook

* Simplified and added XAI

* Fix conflicts

* Ran linter

* Fixed batch prediction request explanation

notebooks/official/automl/sdk_automl_tabular_forecasting_batch.ipynb

@@ -749,22 +749,15 @@
       "source": [
         "### Make the batch prediction request\n",
         "\n",
-        "Now that your Model resource is trained, you can make a batch prediction by invoking the batch_predict() method, with the following parameters:\n",
+        "Now that your Model resource is trained, you can make a batch prediction by invoking the batch_predict() method using a BigQuery source and destination, with the following parameters:\n",
         "\n",
         "- `job_display_name`: The human readable name for the batch prediction job.\n",
-        "- `gcs_source`: Google Cloud Storage URI(-s) to your instances to run batch prediction on.\n",
-        "- `bigquery_source`: BigQuery URI to a table, up to 2000 characters long.\n",
-        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction resuls.\n",
-        "- `machine_type`: The type of machine for running batch prediction on dedicated resources.\n",
-        "- `instances_format`: The format for the input instances, either 'csv' or 'jsonl'. Defaults to 'jsonl'.\n",
-        "- `predictions_format`: The format for the output predictions, either 'csv' or 'jsonl' or 'bigquery'. Defaults to 'jsonl'.\n",
-        "- `starting_replica_count`: The number of machine replicas used at the start of the batch operation. If not set, Vertex AI decides starting number, not greater than `max_replica_count`. Only used if `machine_type` is set.\n",
-        "- `max_replica_count`: The maximum number of machine replicas the batch operation may be scaled to. Only used if `machine_type` is set. Defaults to 10.\n",
-        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete.\n",
-        "\n",
-        "Create a bigquery dataset to store the batch prediction results.\n",
-        "\n",
-        "**Note:** This dataset is only used when the batch prediction job is run with BigQuery options. "
+        "- `bigquery_source`: BigQuery URI to a table, up to 2000 characters long. For example: `bq://projectId.bqDatasetId.bqTableId`\n",
+        "- `bigquery_destination_prefix`: The BigQuery dataset or table for storing the batch prediction resuls.\n",
+        "- `instances_format`: The format for the input instances. Since a BigQuery source is used here, this should be set to `bigquery`.\n",
+        "- `predictions_format`: The format for the output predictions, `bigquery` is used here to output to a BigQuery table.\n",
+        "- `generate_explanations`: Set to `True` to generate explanations.\n",
+        "- `sync`: If set to True, the call will block while waiting for the asynchronous batch job to complete.git "
       ]
     },
     {
@@ -825,15 +818,10 @@
         "batch_prediction_job = model.batch_predict(\n",
         "    job_display_name=f\"iowa_liquor_sales_forecasting_predictions_{TIMESTAMP}\",\n",
         "    bigquery_source=PREDICTION_DATASET_BQ_PATH,\n",
-        "    # instances_format=\"bigquery\",\n",
-        "    instances_format=\"jsonl\",\n",
-        "    # bigquery_destination_prefix=batch_predict_bq_output_uri_prefix,\n",
-        "    gcs_destination_prefix=BUCKET_URI,\n",
-        "    machine_type=\"n1-standard-4\",\n",
-        "    starting_replica_count=1,\n",
-        "    max_replica_count=1,\n",
-        "    # predictions_format=\"bigquery\",\n",
-        "    predictions_format=\"csv\",\n",
+        "    instances_format=\"bigquery\",\n",
+        "    bigquery_destination_prefix=batch_predict_bq_output_uri_prefix,\n",
+        "    predictions_format=\"bigquery\",\n",
+        "    generate_explanation=True,\n",
         "    sync=False,\n",
         ")\n",
         "\n",
@@ -868,14 +856,9 @@
         "id": "get_batch_prediction:mbsdk,forecast"
       },
       "source": [
-        "### Get the predictions\n",
+        "### Get the predictions and explanations\n",
         "\n",
-        "Next, get the results from the completed batch prediction job.\n",
-        "\n",
-        "The results are written to the Cloud Storage output bucket you specified in the batch prediction request. You call the method iter_outputs() to get a list of each Cloud Storage file generated with the results. Each file contains one or more prediction requests in a CSV format:\n",
-        "\n",
-        "- CSV header + predicted_label\n",
-        "- CSV row + prediction, per prediction request"
+        "Next, get the results from the completed batch prediction job and print them out. Each result row will include the prediction and explanation."
       ]
     },
     {
@@ -886,21 +869,8 @@
       },
       "outputs": [],
       "source": [
-        "import tensorflow as tf\n",
-        "\n",
-        "bp_iter_outputs = batch_prediction_job.iter_outputs()\n",
-        "\n",
-        "prediction_results = list()\n",
-        "for blob in bp_iter_outputs:\n",
-        "    if blob.name.split(\"/\")[-1].startswith(\"prediction\"):\n",
-        "        prediction_results.append(blob.name)\n",
-        "\n",
-        "tags = list()\n",
-        "for prediction_result in prediction_results:\n",
-        "    gfile_name = f\"gs://{bp_iter_outputs.bucket.name}/{prediction_result}\"\n",
-        "    with tf.io.gfile.GFile(name=gfile_name, mode=\"r\") as gfile:\n",
-        "        for line in gfile.readlines():\n",
-        "            print(line)"
+        "for row in batch_prediction_job.iter_outputs():\n",
+        "    print(row)"
       ]
     },
     {