Getting to know IBM VPC API
Introduction:
"The world is changed. I feel it in the water. I feel it in the earth. I smell it in the air. Much that once was is lost, for none now live who remember it."
Those with a love for fantasy will surely recognize the quote above, in that quote "Galadriel" is describing the situation which led to the world that became as it is now. Though we are not living in a world of goblins and orcs the world has indeed changed exactly like the description above.
Those old enough can remember that once you were buying software and getting boxes of hard media which you later install on the machines your organization has dedicated for that purpose. for most organizations handling their day to day IT operation is a task which is not easy especially when their business domain lies somewhere else (sometimes quite far away from the IT business), so the current trend for the "new world order" is managing the organization's IT infrastructure by a trusted partner using one of the cloud providers (sometimes several ones).
IBM has made major efforts to shift its business to the cloud, and IBM is progressing steadily to become a leader within the hybrid cloud space. As part of the IBM VPC development for hybrid cloud, I can see more and more clients enlisting to the IBM cloud and moving their workloads to the IBM Cloud.
Typically when you start working with IBM hybrid cloud you would want to create some objects, you can of course use the user interface to do that, but using the user interface although very comfortable has the shortcoming of having an actual user to be active behind the keyboard to fill out all of the required fields. so I am guessing that most of the clients would rather use a scripting development language to manage all the objects within their VPC's space.
Luckily IBM hybrid cloud provides an excellent API just for that case, enter IBM API for VPC IBM REST API
The purpose of this blog would be to explain the basic logic behind the IBM API for VPC and do some basic hands-on training to get you started with IBM cloud for VPC.
please note that the API examples within this blog are "partial" and are also subject to constant change.
A little bit of architecture:
Every IBM cloud data center installation is called a multi-zone region or "MZR", This means that in each IBM cloud installation there are multiple "zones", each zone is equivalent to a "large building" which contains the compute, storage, network hardware, and software which comprises the "IBM cloud", these buildings are separate to make sure if a disaster occurs (flood, tsunami, etc..) only part of the cloud zones will be affected. and the client's business-critical missions can continue to function uninterrupted. Any resources within the cloud could be either regional resources, these resources exist in all of the zones (span all zones). such as a VPC for example. there are also zonal resources that exist only within one zone, a subnet for example.
Getting things ready:
To start scripting for IBM Virtual Private Cloud you will need to start by installing the IBM cloud plugin on your machine :
Detailed instructions on how to install the IBM Cloud CLI: IBM Cloud CLI
Lets set some environment variables :
1. ResourceGroupID:
Execute the following command :
ibmcloud resource groups
export ResourceGroupID= <your_resource_group_id>
2. IAM token:
export token="$(ibmcloud iam oauth-tokens | awk '{ print $4 }')"; echo $token
3. Version :
export version=2020-06-02
4. API endpoint (based on your region):
api_endpoint="https://us-south.iaas.cloud.ibm.com"
The logic behind IBM API for VPC
IBM VPC REST API has 6 types of calls, which should cover all the lifecycle operations needed to manage VPC resources
- POST
- DELETE
- GET
- GET /{Resource}
- PATCH
- PUT
POST Calls
A POST call typically indicates that we want to create a new resource for VPC, some examples of those resources could be a VPC, Network Acl, Security Group, Flow Log collector, and so on.
The response of the POST call should provide anything required to work on that resource at a later time.
Let's take a closer look at a sample POST call, which creates a VPC
curl -X POST -sH "Authorization:${token}" "$api_endpoint/v1/vpcs?version=$version&generation=2" -d '{"name": "myvpc1","resource_group": {"id": "'$ResourceGroupId'"}}' | jq
Sample response :
{
"id": "r134-efe0c24b-c89d-424b-94d9-f6e12856794a",
"crn": "crn:v1:staging:public:is:us-south:a/<accountid>::vpc:r134-efe0c24b-c89d-424b-94d9-f6e12856794a",
"href": "<baseuri>/v1/vpcs/r134-efe0c24b-c89d-424b-94d9-f6e12856794a",
"name": "myvpc1",
"status": "available",
Important sections in the response:
Resource Identifier : (ID/CRN/href): every VPC resource in IBM cloud has a unique identifier, this identifier has three different "aspects", each of the main resources can be located and accessed through the API using the "id", "CRN", or "href" properties, not all of the resources have all three identifiers but most of them have at least two.
The resource Identifier itself contains two sections
zone/region identifier - it indicates in what region or zone the resource exists "r134" in the case above. for more details about this please see the "introduction" section.
UUID - a unique identifier that exists through all the stages of the resource lifecycle.
Name: the name of the resource, which is mostly a user-given name, must be unique within the scope of the IBM cloud, in several cases when the name has not been provided by the user as input the system will automatically generate a name for that resource, which in most cases can be changed anytime later using a PATCH call.
Status: indicated the lifecycle status of the object which has just been created, in many cases the status would be "available", but on some of the "heavier" resources where a longer provisioning period may be required the status would be more like "create_pending" which means that the provisioning process has started but has not yet completed. the status is of course a good point to start to investigate any issues with the object at hand.
DELETE calls
Delete calls are the exact opposite of POST calls, and typically they mean we want to remove a single resource from our current resource portfolio. A delete call will always delete a single resource.
Delete calls typically have various checks in place to make sure that the user cannot delete a resource which has resources attached to it and is required for its correct function, for example, a user cannot delete a VPC that still contains subnets.
In order to delete the VPC, the DELETE/<subnet_id> call is required to be executed prior to the actual call to the DELETE/<vpc_id> call.
On the other hand, there are resources that are considered "subresources" of a major resource. once deleting a major resource, all sub-resources of that major resource are deleted implicitly. for example, deleting a VPC will implicitly delete the flow log collector for that VPC (if such exists).
Succesful DELETE calls do not return any content, but they do return a successful response of HTTP-204 (Deleted) code. if the delete operation was not successful an error code with a description of the error will be returned.
Sample DELETE call - delete VPC :
curl -sS -X DELETE -H "Authorization: $token" $api_endpoint/v1/vpcs/<vpc_id>?version=$version&generation=2 | jq
GET calls
GET calls can have either one of two forms, a GET call without a resource ID, this type of call is considered a LIST call. meaning that all resources need to be returned in a listed manner back to the client.
The List operation can return either one of two forms:
- Simple list - all resources are listed one by one, will typically be the case for small lists
- Paginated list - for long resource lists, it makes no sense returning all of the resources in a single bulk, therefore a page by page iterator pattern is used to return the data, the page size is set by the client call, the default page size is set 50.
Sample List calls :
List VPC call: paginated response expected :
curl -sS -X GET -H "Authorization: $token" $api_endpoint/v1/vpcs?version=2020-06-02 | jq
output :
{
"limit": 50,
"first": {
"href": "<baseuri>/v1/vpcs?limit=50"
},
"total_count": 1,
"vpcs": [
{
"id": "r134-b004970f-ffcd-465b-9b68-b73a735c8151",
"crn": "crn:v1:staging:public:is:us-south:a/<accountid>::vpc:r134-b004970f-ffcd-465b-9b68-b73a735c8151",
"href": "<baseuri>/v1/vpcs/r134-b004970f-ffcd-465b-9b68-b73a735c8151",
In order to traverse through the paginated list, we would need to provide the "start" parameter, this parameter will indicate on which of the resource id's the listing of the first page begins.
example:
curl -X GET -sH "Authorization:${token}" "$api_endpoint/v1/vpcs?version=$version&start=$VpcId" | jq
List routing table routes: non paginated response expected:
curl -sS -X GET -H "Authorization: $token" $api_endpoint/v1/vpcs/$VpcId/routing_tables/$DefRoutingTableId?version=$version | jq
output:
{
"id": "r134-9826a943-405e-49ec-b3bb-ec031338fcb5",
"href": "<baseuri>/v1/vpcs/r134-b004970f-ffcd-465b-9b68-b73a735c8151/routing_tables/r134-9826a943-405e-49ec-b3bb-ec031338fcb5",
"name": "sixteen-clamshell-feedable-thinness",
"resource_type": "routing_table",
"created_at": "2020-10-19T09:45:38Z",
Please note that the list operation might not return all of the data which the target resource might contain, therefore in some cases when we need the full information we will need to call the GET/<resource_id> operation.
GET/<resource_id> calls
As opposed to a list operation. which is typically done when we do not have an identifier which we would like to investigate, the GET /<resource_id> call is done to return a single entity, the returned entity is fully "loaded" and is guaranteed to return all the information which is relevant to work with the object at hand.
The GET /<resource_id> is a very efficient call both on CPU and network resources, as it returns only the "minimal" required data without the overhead of a list operation. so as a rule of thumb to make our calls more efficient we should prefer using GET calls instead of LIST calls.
It's also very efficient on the server-side as no data is manipulated during the course of the call.
Sample get call - Get VPC :
curl -X GET -sH "Authorization:${token}" "$api_endpoint/v1/vpcs/$VpcId?version=2019-10-03" | jq
response:
{
"id": "r134-b004970f-ffcd-465b-9b68-b73a735c8151",
"crn": "crn:v1:staging:public:is:us-south:a/<accountid>::vpc:r134-b004970f-ffcd-465b-9b68-b73a735c8151",
"href": "<baseuri>/v1/vpcs/r134-b004970f-ffcd-465b-9b68-b73a735c8151",
"name": "myvpc",
PATCH Calls
In some cases, we might want to create a resource and defer some of its configuration parameters to be specified at a later time.
This is most convenient when you want to start something quickly and at a later time augment your configuration.
The patch operation allows some configuration parameters to be modified at a later time. It's important to notice that not all of the configuration parameters can be modified later, some configurations are immutable after the creation of the target resource so, to change the configuration you will need to delete the object and re-create it using the new parameters.
Patch calls could provide a relatively simple service as changing the name of the resource, and on the other hand, the call can execute something much more complex like changing the target of the resource or even moving it to another availability zone.
sample patch call - changes vpc name :
curl -k -sS -X PATCH -H "Authorization: ${token}" $api_endpoint/v1/vpcs/$VpcId?version=$version -d '{ "name": "newname" }' | jq
response:
{
"id": "r134-b004970f-ffcd-465b-9b68-b73a735c8151",
"crn": "crn:v1:staging:public:is:us-south:a/<accountid>::vpc:r134-b004970f-ffcd-465b-9b68-b73a735c8151",
"href": "<baseuri>/v1/vpcs/r134-b004970f-ffcd-465b-9b68-b73a735c8151",
"name": "newname",
PUT Calls
The PUT call is currently the rarest call within the IBM VPC API, we use the PUT call in order to replace a resource or a collection of resources in a single PUT call.
as opposed to patch where the target resource stays the same (same unique identifier), and only part of the resource is being modified. the PUT call replaces a resource entirely (new identifier/s)
sample call - replaces the routing table within a subnet :
curl -X PUT -sH "Authorization:${token}" $api_endpoint/v1/subnets/$SubnetId/routing_table?version=$version -d '{"id": "'$RoutingTableId'" }' | jq
response (modified routing table):
{
"id": "r134-9826a943-405e-49ec-b3bb-ec031338fcb5",
"href": "<baseuri>/v1/vpcs/r134-b004970f-ffcd-465b-9b68-b73a735c8151/routing_tables/r134-9826a943-405e-49ec-b3bb-ec031338fcb5",
"name": "sixteen-clamshell-feedable-thinness",
"subnets": [
{
"id": "7392-04925f2c-9559-4108-866e-f2ad4e8a8d89",
"crn": "crn:v1:staging:public:is:us-south-3:a/<accountid>::subnet:7392-04925f2c-9559-4108-866e-f2ad4e8a8d89",
"href": "<baseuri>/v1/subnets/7392-04925f2c-9559-4108-866e-f2ad4e8a8d89",
"name": "subnet-1",
conclusion:
During the progress of this post, I tried to explain the logic which comprises the IBM API for VPC (nextgen), I believe it's very beneficial for anyone who tries to use scripting for IBM API for VPC to get to know a little more than just to issue the commands themselves.
Although this post was quite long it does not do a deep dive into any one resource on the API, instead, it tries to establish the basic concepts laid by the API.
