Functioning In The Cloud

Welcome back! We are just done with our GitHub TODO issue creator thing. This time around we’re going to go through the steps to deploy it as a Google Cloud Function. To do this we’ll need to alter the code. But before we dive into the code itself we’ll need to get setup to use Go modules.

Go Mod

First, we’ll need to get our mod file ready. You may or may not already have modules enabled. If you do you should be able to leave off the GO111MODULE=on part of the command.

GO111MODULE=on go mod issuer

Now, we need to make sure we’ve got all the modules we need.

➜  issuer GO111MODULE=on go mod tidy
go: finding github.com/google/go-github/v25/github latest
go: finding golang.org/x/oauth2 latest
go: downloading google.golang.org/appengine v1.4.0
go: extracting google.golang.org/appengine v1.4.0
go: downloading github.com/golang/protobuf v1.2.0
go: extracting github.com/golang/protobuf v1.2.0
go: downloading golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6
go: extracting golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6

Looking good so far! Next we want to vendor our dependancies. This copies the modules we downloaded in the previous step into our code directory.

GO111MODULE=on go mod vendor

We should now see the vendor directory.

➜  issuer ls
go.mod  go.sum  main.go vendor

Alterations

Now let’s make a few changes to our code to get it ready for deploying as a Cloud Function. We’ll be starting off the same as last time. You may notice that I’ve removed the ability to query /status, it is not longer needed.

package issuer

import (
  "context"
  "encoding/json"
  "fmt"
  "log"
  "net/http"
  "os"
  "strconv"

  "github.com/google/go-github/v25/github"
  "github.com/shindakun/envy"
  "golang.org/x/oauth2"
)

Note that I’ve pulled out our constants and replaced them will variables. This allows us to make the code a bit more generic and so others can use it. We’ll cover how we are setting up these variables a bit later on. Note, some people may not like the use of global variables here. Since our function is so small and we have a good idea what’s happening in each step it’s OK to use them for now.

Of course if you’d rather not have globals feel free to remove them. You can hard code as constants or load them inside our main function call.

// RepoOwner is the owner of the repo we want to open an issue in
var RepoOwner string

// IssueRepo is the repo we want to open this new issue in
var IssueRepo string

// ProjectColumn is the TODO column number of the project we want to add the issue to
var ProjectColumn int64

// Token is the GitHub Personal Access Token
var Token string

// Secret is used to validate payloads
var Secret string

// Payload of GitHub webhook
type Payload struct {
  Action string `json:"action"`
  Issue  struct {
    URL           string `json:"url"`
    RepositoryURL string `json:"repository_url"`
    Number        int    `json:"number"`
    Title         string `json:"title"`
    Body          string `json:"body"`
  } `json:"issue"`
  Repository struct {
    Name string `json:"name"`
  } `json:"repository"`
}

I’ve also changed handleWebhook() to HandleWebhook(). Exporting the function is what allows us to call it as the cloud function.

func HandleWebhook(res http.ResponseWriter, req *http.Request) {
  var Payload Payload
  defer req.Body.Close()

  p, err := github.ValidatePayload(req, []byte(Secret))
  if err != nil {
    http.Error(res, "bad request: "+err.Error(), 400)
    log.Printf("bad request: %v", err.Error())
    return
  }

As @kunde21 pointed out last week we are better off using json.Unmarshal() here. This also allows us to remove the imports for bytes and io/ioutil.

  err = json.Unmarshal(p, &Payload)
  if err != nil {
    http.Error(res, "bad request: "+err.Error(), 400)
    log.Printf("bad request: %v", err.Error())
    return
  }

  err = createNewIssue(&Payload)
  if err != nil {
    log.Printf("bad request: %v", err.Error())
    return
  }
}

func createNewIssue(p *Payload) error {
  log.Printf("Creating New Issue.\n")
  log.Printf("  Name: %#v\n", p.Repository.Name)
  log.Printf("  Title: %#v\n", p.Issue.Title)
  log.Printf("  Body: %#v\n", p.Issue.Body)
  log.Printf("  URL: %#v\n", p.Issue.URL)

  ctx := context.Background()
  ts := oauth2.StaticTokenSource(
    &oauth2.Token{AccessToken: Token},
  )
  tc := oauth2.NewClient(ctx, ts)

  client := github.NewClient(tc)

  title := fmt.Sprintf("[%s] %s", p.Repository.Name, p.Issue.Title)
  body := fmt.Sprintf("%s\n%s/%s#%d", p.Issue.Body, RepoOwner, p.Repository.Name, p.Issue.Number)

  issue := &github.IssueRequest{
    Title: &title,
    Body:  &body,
  }

  ish, _, err := client.Issues.Create(ctx, RepoOwner, IssueRepo, issue)
  if err != nil {
    log.Printf("error: %v", err)
    return err
  }

  id := *ish.ID
  card := &github.ProjectCardOptions{
    ContentID:   id,
    ContentType: "Issue",
  }

  _, _, err = client.Projects.CreateProjectCard(ctx, ProjectColumn, card)
  if err != nil {
    log.Printf("error: %v", err)
    return err
  }

  return nil
}

Now we get to one of the biggest changes. Since the cloud function is going to call HandleWebhook(), we no longer need our main(). But that presents an issue, we have some environment variables we want to use. We could load them in the HandleWebhook() call it’s more appropriate to make use of Go’s init().

init() runs before main (or our handler in this case), which allows us to load our variables as normal.

func init() {
  log.Println("Issuer")
  var err error
  Token, err = envy.Get("GITHUBTOKEN")
  if err != nil || Token == "" {
    log.Printf("error: %v", err)
    os.Exit(1)
  }

  Secret, err = envy.Get("SECRET")
  if err != nil || Secret == "" {
    log.Printf("error: %v", err)
    os.Exit(1)
  }

  RepoOwner, err = envy.Get("REPOOWNER")
  if err != nil || RepoOwner == "" {
    log.Printf("error: %v", err)
    os.Exit(1)
  }

  IssueRepo, err = envy.Get("ISSUEREPO")
  if err != nil || IssueRepo == "" {
    log.Printf("error: %v", err)
    os.Exit(1)
  }

ProjectColumn requires a bit more setting up since I never extended envy to return ints. So we need to convert to an int64 before we can use the column numeric ID to create our TODO card on the kanban board.

  n, err := envy.Get("PROJECTCOLUMN")
  if err != nil || n == "" {
    log.Printf("error: %v", err)
    os.Exit(1)
  }
    ProjectColumn, err = strconv.ParseInt(n, 10, 64)
      if err != nil || ProjectColumn == 0 {
    log.Printf("error: %v", err)
    os.Exit(1)
  }
}

Go To The Cloud

I am going to assume that you have the Google Cloud command line tools installed and a project set up. If you do not Google has some very good tutorials. Checkout https://cloud.google.com/functions/docs/quickstart for more

Note: You wouldn’t want to have the secret and the token deployed like this in production. You would instead want to use Cloud KMS or Runtime Configurator. I’m living dangerously.

➜  issuer gcloud functions deploy issuer --entry-point HandleWebhook --runtime go111 --trigger-http --memory=128MB --set-env-vars SECRET=secret,GITHUBTOKEN=token,REPOOWNER=shindakun,ISSUEREPO=to,PROJECTCOLUMN=5647145

Deploying function (may take a while - up to 2 minutes)...done.
availableMemoryMb: 128
entryPoint: HandleWebhook
environmentVariables:
  GITHUBTOKEN: token
  ISSUEREPO: to
  PROJECTCOLUMN: '5647145'
  REPOOWNER: shindakun
  SECRET: secret
httpsTrigger:
  url: https://us-east1-golang-slackbot.cloudfunctions.net/issuer
labels:
  deployment-tool: cli-gcloud
name: projects/golang-slackbot/locations/us-east1/functions/issuer
runtime: go111
serviceAccountEmail: golang-slackbot@appspot.gserviceaccount.com
sourceUploadUrl: https://storage.googleapis.com/gcf-upload-us-central1-eeb5af0e-fd09-4fe7-30851592ebba/bcc11c6f-55fc-4d73-864a-6a89813206a6.zip?GoogleAccessId=service-84452958925@gcf-gserviceaccount.com&Expires=1561244032&Signature=QX%2BKy5j6YTA6%D%3D
status: ACTIVE
timeout: 60s
updateTime: '2019-06-22T22:24:37Z'
versionId: '1'

Next time

And there we have it! We’re now live and update any source repos to use our trigger URL. That makes this stage of our issuer complete. New issues will appear in our target repo and on the TODO section of the kanban board!

What should we tackle next time? I’ll have to take a look at the kanban board and see if any ideas jump out at me. Until then feel free to let me know if you spot something to refactor.

You can find the code for this and most of the other Attempting to Learn Go posts in the repo on GitHub.

{% github shindakun/atlg %}

Enjoy this post?
How about buying me a coffee?