fix: numpy bool

notebooks/official/workbench/spark/spark_ml.ipynb

@@ -127,25 +127,6 @@
         "You can use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
       ]
     },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "XoEqT2Y4DJmf"
-      },
-      "source": [
-        "## Before you begin\n",
-        "\n",
-        "### Set up your Google Cloud project:\n",
-        "\n",
-        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you create an account, you receive a $300 credit towards to your compute and storage costs.\n",
-        "\n",
-        "1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
-        "\n",
-        "1. [Enable the BigQuery API, Notebooks API, Vertex AI API, and Dataproc API](https://console.cloud.google.com/flows/enableapi?apiid=bigquery.googleapis.com,notebooks.googleapis.com,aiplatform.googleapis.com,dataproc&_ga=2.209429842.1903825585.1657549521-326108178.1655322249)\n",
-        "\n",
-        "**Note**: Jupyter runs lines prefixed with `!` as shell commands, and inserts the value of Python variables prefixed with `$` into the commands."
-      ]
-    },
     {
       "cell_type": "markdown",
       "metadata": {
@@ -157,25 +138,6 @@
         "Install the following packages:"
       ]
     },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "172533a994ad"
-      },
-      "outputs": [],
-      "source": [
-        "import os\n",
-        "\n",
-        "# Google Cloud Notebook requirements:\n",
-        "IS_GOOGLE_CLOUD_NOTEBOOK = os.path.exists(\"/opt/deeplearning/metadata/env_version\")\n",
-        "\n",
-        "USER_FLAG = \"\"\n",
-        "# Google Cloud Notebook dependencies to be installed with '--user':\n",
-        "if IS_GOOGLE_CLOUD_NOTEBOOK:\n",
-        "    USER_FLAG = \"--user\""
-      ]
-    },
     {
       "cell_type": "code",
       "execution_count": null,
@@ -184,6 +146,8 @@
       },
       "outputs": [],
       "source": [
+        "import os\n",
+        "\n",
         "if os.getenv(\"IS_TESTING\"):\n",
         "    \"\"\"\n",
         "    Since the testing suite doesn't support testing on Dataproc clusters,\n",
@@ -205,7 +169,7 @@
         "    ! echo $JAVA_HOME\n",
         "\n",
         "    # Install PySpark and other packages.\n",
-        "    ! pip install {USER_FLAG} pyspark==$PYSPARK_VER geopandas pyarrow rtree seaborn -q"
+        "    ! pip install pyspark==$PYSPARK_VER geopandas pyarrow rtree seaborn numpy==1.19.5 -q"
       ]
     },
     {
@@ -280,80 +244,46 @@
       "source": [
         "### Set your project ID\n",
         "\n",
-        "Run the following cell to get your project ID."
+        "**If you don't know your project ID**, try the following:\n",
+        "* Run `gcloud config list`.\n",
+        "* Run `gcloud projects list`.\n",
+        "* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
       ]
     },
     {
       "cell_type": "code",
       "execution_count": null,
       "metadata": {
-        "id": "4c7fe49d5dac"
+        "id": "set_project_id"
       },
       "outputs": [],
       "source": [
-        "import os\n",
+        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
         "\n",
-        "PROJECT_ID = \"\"\n",
-        "\n",
-        "# Get your Google Cloud project ID from gcloud\n",
-        "if not os.getenv(\"IS_TESTING\"):\n",
-        "    shell_output = !gcloud config list --format 'value(core.project)' 2>/dev/null\n",
-        "    PROJECT_ID = shell_output[0]\n",
-        "    print(\"Project ID: \", PROJECT_ID)"
+        "# Set the project id\n",
+        "! gcloud config set project {PROJECT_ID}"
       ]
     },
     {
       "cell_type": "markdown",
       "metadata": {
-        "id": "XoEqT2Y4DJmf"
-      },
-      "source": [
-        "If the previous command has no output, copy your project ID from the project selector in the [Google Cloud console](https://console.cloud.google.com/). Insert the ID in the `[your-project-id]` placeholder, then run the following command:"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "282bc9bcbffe"
-      },
-      "outputs": [],
-      "source": [
-        "if not PROJECT_ID or PROJECT_ID == \"\":\n",
-        "    PROJECT_ID = \"[your-project-id]\"  # @param {type: \"string\"}"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "8eef423a7e5f"
-      },
-      "outputs": [],
-      "source": [
-        "! gcloud config set project $PROJECT_ID -q"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "XoEqT2Y4DJmf"
-      },
-      "source": [
-        "### Create a Cloud Storage bucket\n",
-        "\n",
-        "The machine learning model created in this tutorial is stored in a Cloud Storage bucket."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "XoEqT2Y4DJmf"
+        "id": "region"
       },
       "source": [
         "#### Region\n",
         "\n",
-        "Before creating a Cloud Storage bucket, redefine the `REGION` variable (when you change the notebook kernel, previously set variables are deleted)."
+        "You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "2dw8q9fdQEH5"
+      },
+      "outputs": [],
+      "source": [
+        "REGION = \"us-central1\""
       ]
     },
     {
@@ -405,58 +335,101 @@
     {
       "cell_type": "markdown",
       "metadata": {
-        "id": "XoEqT2Y4DJmf"
+        "id": "gcp_authenticate"
       },
       "source": [
-        "Replace the `[your-bucket-name]` placeholder with the name of your Cloud Storage bucket. The name must be unique across all Cloud Storage buckets."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "78c2bc5c2a35"
-      },
-      "outputs": [],
-      "source": [
-        "BUCKET_NAME = \"[your-bucket-name]\"  # @param {type:\"string\"}\n",
-        "BUCKET_URI = f\"gs://{BUCKET_NAME}/\"\n",
+        "### Authenticate your Google Cloud account\n",
         "\n",
-        "# If you do not specify a bucket name, it will be created based on your project ID and the UUID.\n",
-        "if not BUCKET_NAME or BUCKET_NAME == \"\" or BUCKET_NAME == \"[your-bucket-name]\":\n",
-        "    BUCKET_NAME = f\"{PROJECT_ID}{UUID}\"\n",
-        "    BUCKET_URI = f\"gs://{BUCKET_NAME}/\""
+        "Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
+        "\n",
+        "**1. Vertex AI Workbench**\n",
+        "* Do nothing as you are already authenticated.\n",
+        "\n",
+        "**2. Local JupyterLab instance, uncomment and run:**"
       ]
     },
     {
       "cell_type": "code",
       "execution_count": null,
       "metadata": {
-        "id": "17be4f420fd2"
+        "id": "ce6043da7b33"
       },
       "outputs": [],
       "source": [
-        "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_URI"
+        "# ! gcloud auth login"
       ]
     },
     {
       "cell_type": "markdown",
       "metadata": {
-        "id": "XoEqT2Y4DJmf"
+        "id": "0367eac06a10"
       },
       "source": [
-        "Confirm your access to the Cloud Storage bucket by displaying the bucket's metadata:"
+        "**3. Colab, uncomment and run:**"
       ]
     },
     {
       "cell_type": "code",
       "execution_count": null,
       "metadata": {
-        "id": "ad8f9c4923ed"
+        "id": "21ad4dbb4a61"
       },
       "outputs": [],
       "source": [
-        "! gsutil ls -L -b $BUCKET_URI"
+        "# from google.colab import auth\n",
+        "# auth.authenticate_user()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "c13224697bfb"
+      },
+      "source": [
+        "**4. Service account or other**\n",
+        "* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "bucket:mbsdk"
+      },
+      "source": [
+        "### Create a Cloud Storage bucket\n",
+        "\n",
+        "Create a storage bucket to store intermediate artifacts such as datasets."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "bucket"
+      },
+      "outputs": [],
+      "source": [
+        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "autoset_bucket"
+      },
+      "source": [
+        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "91c46850b49b"
+      },
+      "outputs": [],
+      "source": [
+        "! gsutil mb -l $REGION -p $PROJECT_ID $BUCKET_URI"
       ]
     },
     {
@@ -478,6 +451,8 @@
       },
       "outputs": [],
       "source": [
+        "import os\n",
+        "\n",
         "import matplotlib.pyplot as plt\n",
         "import numpy as np\n",
         "import pandas as pd\n",
@@ -1531,9 +1506,11 @@
       },
       "outputs": [],
       "source": [
+        "import os\n",
+        "\n",
         "DELETE_BUCKET = True\n",
         "\n",
-        "if DELETE_BUCKET:\n",
+        "if DELETE_BUCKET or os.getenv(\"IS_TESTING\"):\n",
         "    ! gsutil rm -r $BUCKET_URI"
       ]
     },