guile-aws

Working with the AWS API in Guile

I stumbled upon an interesting Gnu Guile experiment: guile-aws. It's more than just an implementation of the AWS API in Scheme; it's an exposition of the power of the Guile compiler tower. "Guile defines a tower of languages, starting at Scheme and progressively simplifying down to languages that resemble the VM instruction set." One can extend Guile by defining a new "language" that compiles-down to Scheme (and is then compiled-down further as usual). This is what the author (one Ricardo Wurmus) has done.

He's pulled-down JSON documents from the AWS Javascript SDK, each of which describes a particular AWS API (S3, ECS, Route53 and so forth). He's then defined Scheme code that produces Scheme functions for each endpoint described therein. In other words, a fragment of JSON like:

"ListObjectsV2": {
      "name": "ListObjectsV2",
      "http": {
        "method": "GET",
        "requestUri": "/{Bucket}?list-type=2"
      },
      "input": {
        "shape": "ListObjectsV2Request"
      },
      "output": {
        "shape": "ListObjectsV2Output"
      },
      "errors": [
        {
          "shape": "NoSuchBucket"
        }
      ],
      "documentation": "<p>Returns some or all (up to 1,000) of the objects in a bucket with each request..."
    }
...
"ListObjectsV2Request": {
      "type": "structure",
      "required": [
        "Bucket"
      ],
      "members": {
        "Bucket": {
          "shape": "BucketName",
          "documentation": "<p>Bucket name to list..."
          "contextParam": {
            "name": "Bucket"
          },
          "location": "uri",
          "locationName": "Bucket"
        },
        "Prefix": {
          "shape": "Prefix",
          "documentation": "<p>Limits the response to keys that begin with the specified prefix.</p>",
          "location": "querystring",
          "locationName": "prefix"
        },
        ...
      }
    }

results in a Scheme function that can be invoked as:

(ListObjectsV2 #:Bucket "my-bucket" #:Prefix "some-prefix")

Here's his e-mail to the guile-user list announcing the project. He hosts the project at elephly.net, where it appears dormant. I "forked" it to Github & made a few updates:

I fixed a small bug in the request signing implementation
I updated the JSON documents describing the API from v2.680.0 to v2.1499.0 of the AWS Javascript SDK

All that said, for my particular application (grabbing some files from S3), the implementation didn't work. There were a few problems:

hitting the documented S3 endpoint results in a 301 (Moved Permanently) which the package isn't ready to handle
more seriously, it's not clear to me how the JSON documents are to be precisely interpreted (big surprise), but the author's interpretation matches neither the documentation nor my empirical experience: the S3 endpoints I'm hitting (ListObjectsV2 & GetObject) do not take an XML-encoded request body as indicated by the JSON: they encode their parameters in the query string

If AWS were to provide a machine-readable, accurate description of their APIs, it would enable automated SDK generation in numerous languages (lol… I crack myself up).

On the other hand, the package does provide a Scheme implementation of the SigV4 request signing protocol. That, along with my reading of the code, has equipped me to hit the S3 API from Scheme on my own.

11/18/23 17:17

Unwound Stack

guile-aws