Using Google Cloud Function to generate data for Machine Learning model

Prerequisite: Deploy cloud function on Google Cloud Platform

Do you search for data to train your model online? What if we tell that you can generate your own data in just a few lines on code?

All you need is a Google Cloud Platform account and know how to deploy simple code to Cloud Function. We will be using Google Sheets to store the data. You can use Cloud SQL or Google Cloud Storage Bucket or Firebase to store the data. All you need to do is enable the necessary APIs.

Enabling the APIs and creating the credentials:

1. Go to Marketplace in Cloud Console.
2. Click on ENABLE APIS AND SERVICES
3. Then Search for Google Drive API and enable it
4. Then go to the Credentials tab on the left navigation bar on the screen.
5. Then click on Create Credentials then select Service Account Key
6. Then create a new service account by giving it a name and set the Role to Editor under the Projects sub-field and keep the key type as JSON and click on Create button. Keep the Downloaded JSON safely.

After all these steps are done your page should look something like this



7. Again go to Dashboard and follow the same steps. This time search for Google Sheets and enable the API.

Creating the Spreadsheet

1. Create a Spreadsheet in Google Sheets
2. The look up the downloaded JSON file for the field client_email and copy that email.
3. Open the newly created spreadsheet and click on the share option and type the paste the client_email there.

So, the boring part is done. Now, lets jump into the code.

Setting up Cloud Functions

1. Create a new Cloud Function and change the Runtime to Python 3.7
2. Go to requirements.txt and delete the boilerplate text and add the following lines to it.

   gspread>=3.1.0
   oauth2client>=4.1.3

3. Now the most important and the best part, writing the code. Delete the entire boilerplate code and paste the following code

   import gspread
   from oauth2client.service_account import ServiceAccountCredentials
   from datetime import datetime
   
   def update(request):
       # getting the variables ready
       data = {
           # your client_json contents as dictionary
       }
   
       request_json = request.get_json()
       request_args = request.args
       temp = ""
   
       if request_json and 'temp' in request_json:
           temp = request_json['temp']
       elif request_args and 'temp' in request_args:
           temp = request_args['temp']
   
       # use creds to create a client to interact with the Google Drive API
       scope = ['https://spreadsheets.google.com/feeds', 'https://www.googleapis.com/auth/drive']
   
       creds = ServiceAccountCredentials.from_json_keyfile_dict(data, scope)
   
       client = gspread.authorize(creds)
   
       # Find a workbook by name and open the first sheet
       # Make sure you use the right name here.
       sheet = client.open("Temperature").sheet1
   
       row = [datetime.now().strftime("% d/% m/% Y % H:% M:% S"), temp]
       index = 2
       sheet.insert_row(row, index)

   Code Explanation –

   -> First, we get the necessary imports. gspread is the library for performing handling of Google Sheets. Then, we are importing oauth2client. This will handle our authentication of the generated credentials. Then, we are importing datetime to log the data correctly with the current time and date.

   -> Coming into the update(request), first we add our credentials in the data dictionary. Then, we store the user request parameters in a variable called request_json and then we are initializing temp and then assigning the ‘temp’ key value into the variable.

   -> We are defining our scope for the authentication. We are using oauth2client to make the necessary authentication with the specified scopes.
   Then, open your Google Spreadsheet by specifying your Sheet name there. Then, we are inserting the date and time of logging and the parameter in the sheet at the specified row in index variable.

4. Now type update in the Function to execute and then click on ddeploy

After that your Cloud Function page should look like this

After this, you need to click on the function name, here function-1


Then, go to trigger tab


After that, note the URL that is shown in there. This is the URL you will send GET request along with the data parameter to add the parameter value to the spreadsheet.

Setting up IoT device
You can use Nodemcu or Arduino for sending the data to Google Sheets, but you will be requiring a WiFi module along with it and ofcourse, Raspberry Pi can also be used. Now, all you have to do is send http request to the Cloud Function URL along with the parameters, here temp or temperature. This would edit the spreadsheet and add the parameter values to the spreadsheet.

With that being said, this is how you can use Cloud Functions to log data to Google Sheets. You can do the same for Cloud SQL or any other means of storage. The stored data can be used as training data for its relevant Machine Learning model.