Jaa

5 Reasons Why You Will Enjoy Using the new Swordfish API

  • Artikkeli

Hello everyone, Taking this opportunity to inform you about one of the projects I am working on is a new datacenter management API called Swordfish. For those of you aware of Microsoft's dedication to standards based management, we continue to collaborate with SNIA and DMTF member companies on this project.

Before We Begin

MEET-UP INVITATION

This is a long post so let me first invite you to an end-user meet-up hosted by SNIA Microsoft Ignite in Atlanta.

  • You planning on attending Ignite this year?
  • Do your roles responsibilities include storage management in the datacenter (SAN (iSCSI/FC/FCoE SAN and NAS storage in a Linux, Windows Server, VMware, Hyper-V, OpenStack private cloud environment)?
  • Want to spend 90 minutes providing feedback to leading industry experts and the creators of Swordfish API?

Then this meet-up is for you. This is an opportunity for SNIA to collect feedback from you about the future of storage management and you get to shape the evolution of the new Swordfish API post v1.0. We have very few seats available so you MUST RSVP to guarantee your spot. At the meet-up you will have an opportunity to sign up for a futures council that SNIA is in the process of forming. You also will have an opportunity to win a Phantom 3.
RSVP

Swordfish at SDC and Microsoft Ignite 

Are you attending SNIA SDC in Santa Clara this week? Pass by the booth and say hello. Several member companies working on Swordfish will be there. I get there tomorrow Wednesday.

Attending Microsoft Ignite? Pass by SNIA SMI booth #2371 and check out the cool Swordfish demos.

Background: Building Blocks for Swordfish

Back to our regularly scheduled program. Swordfish is part of a set of projects centered around the Redfish protocol. DMTF formed the Scalable Platform Management Forum (SPMF) in 2015 and created Redfish API to simplify server management in traditional and cloud datacenters using simple and modern API. In addition to the API, SPMF defined the Redfish protocol with oDATA support to standardize RESTful APIs conventions. SPMF released v1 of the Redfish protocol and API in 2015. Since then SPMF has released 3 updates to the schema. In parallel, two groups started working on extensions to Redfish schema:

  • The Chinook working group developed schema extensions for network switch management
  • SNIA's Scalable Storage Management (SSM) technical working group developed Swordfish extensions for storage management.

Swordfish v1.0

The benefit of extending Redfish schemas stems from the simplicity of Redfish and very low learning curve for managing devices in the datacenter. If you are familiar with using a browser, you can start to explore a devices managed by Redfish. In this blog post I will show you 5 examples in the context of storage management using the Swordfish schema extensions.

Terminology

Term Definition
Redfish Designed to meet the expectations of end users for simple and secure management of modern scalable platform hardware. An open industry standard specification and schema that specifies a RESTful interface and utilizes JSON and OData to help integrate solutions within existing tool chains
Swordfish Extensions to Redfish schema developed by SNIA for storage management. For the purposes of this blog, "managed by Swordfish" refers to a SAN with Swirdfish provider. Keep in mind this is only a small sample of the Swordfish API. It's scope spans beyond SAN into NAS, class of service, file systems, endpoints. We will leave the deep dive for another post.
Provider Network endpoint that supports implements Redfish and Swordfish APIs
Redfish protocol HTTPS is the application protocol that will be used to transport the messages and TCP/IP is the transport protocol. The RESTful interface is a mapping to the message protocol. For simplicity though, we will refer to the RESTful mapping to HTTPS, TCP/IP and other protocol, transport and messaging layer aspects as the Redfish protocol.

Example 1: It's a hypermedia API that is simple to navigate

Disclaimer. The schema in this blog post is based on the v0.9 release of Swordfish. We went ahead and hosted the REST APIs on Mockable.IO so you can test drive the examples yourself. At this point in time, the APIs return sample data. The information in this mockup may change once the final version of Swordfish is released.

For these examples I assume you will use Chrome with the JSONView extension installed.

Let's start by exploring the resources of a SAN managed by Swordfish. In your browser, navigate to

 https://swordfish.mockable.io/redfish/v1

The JSON document you see in the browser lists resources managed by Swordfish and high level information and hosted services of the Swordfish provider implementation.

 {
    @odata.context: "/redfish/v1/$metadata#ServiceRoot",
    @odata.id: "/redfish/v1/",
    @Redfish.Copyright: "Copyright 2014-2016 Distributed Management Task Force, Inc. (DMTF). For the full DMTF copyright policy, see https://www.dmtf.org/about/policies/copyright.",
    @odata.type: "#ServiceRoot.v1_0_2.ServiceRoot",
    Id: "RootService",
    Name: "Root Service",
    RedfishVersion: "1.0.2",
    UUID: "92384634-2938-2342-8820-489239905423",
    Systems: {
        @odata.id: "/redfish/v1/Systems"
        },
    StorageSystems: {
        @odata.id: "/redfish/v1/StorageSystems"
    },
    StorageServices: {
        @odata.id: "/redfish/v1/StorageServices"
    },
    Chassis: {
        @odata.id: "/redfish/v1/Chassis"
    },
    Managers: {
        @odata.id: "/redfish/v1/Managers"
    },
    Tasks: {
        @odata.id: "/redfish/v1/TaskService"
    },
    SessionService: {
        @odata.id: "/redfish/v1/SessionService"
    },
    AccountService: {
        @odata.id: "/redfish/v1/AccountService"
    },
    EventService: {
        @odata.id: "/redfish/v1/EventService"
    },
    Links: {
        Sessions: {
            @odata.id: "/redfish/v1/SessionService/Sessions"
        }
    },
    Oem: { }
}

In this JSON document, StorageSystems and StorageServices represent high level resources introduced by the Swordfish schema extensions. Redfish is a hypermedia API so you can use the @odata.id values to navigate to the resources contained within StorageSystems and StorageServices.

For example, you can take the following @odata.id value

 @odata.id": "/redfish/v1/StorageSystems

and append it to https://swordfish.mockable.io to get the following URI:

 https://swordfish.mockable.io/redfish/v1/StorageSystems

This API gets the list of all StorageSystems managed by this Swordfish endpoint:

 {
    @Redfish.Copyright: "Copyright 2015-2016 SNIA. All rights reserved.",
    @odata.context: "/redfish/v1/$metadata#StorageSystems",
    @odata.id: "/redfish/v1/StorageSystems",
    @odata.type: "#StorageSystemCollection.StorageSystemCollection",
    Name: "Storage System Collection",
    Members@odata.count: 3,
    Members: [
        {
            @odata.id: "/redfish/v1/StorageSystems/Simple"
        },
        {
            @odata.id: "/redfish/v1/StorageSystems/Complex"
        },
        {
            @odata.id: "/redfish/v1/StorageSystems/FileServer"
        }
    ]
}

Example 2: Quickly gain access to detailed information about a resource

To get information about the SIMPLE StorageSystem, use the @odata.id value of the first member in the list above to create the following URI:

 https://swordfish.mockable.io/redfish/v1/StorageSystems/Simple

A StorageSystem in Swordfish represents the set of physical or virtual systems that host one or more StorageServices where you can manage StoragePools, Volumes, and FileSystems.

The following API call gets all the information about the SIMPLE StorageSystems managed by this Swordfish endpoint:

 {
  "@Redfish.Copyright": "Copyright 2015-2016 SNIA. All rights reserved.",
  "@odata.context": "/redfish/v1/$metadata#ComputerSystem.ComputerSystem",
  "@odata.id": "/redfish/v1/StorageSystems/Simple",
  "@odata.type": "#ComputerSystem.1.0.0.ComputerSystem",
  "Id": "Simple",
  "Name": "My storage System",
  "SystemType": "Physical",
  "AssetTag": "free form asset tag",
  "Manufacturer": "Manufacturer Name",
  "Model": "Model Name",
  "SKU": "",
  "SerialNumber": "2M220100SL",
  "PartNumber": "",
  "Description": "Description of server",
  "Identifiers": {
    "UUID": "00000000-0000-0000-0000-000000000000"
  },
  "HostName": "srv344",
  "HostingRole": "StorageServer",
  "Status": {
    "State": "Enabled",
    "Health": "OK",
    "HealthRollUp": "OK"
  },
  "PowerState": "On",
  "Boot": {...},
  "BiosVersion": "Xxxx",
  "ProcessorSummary": {...},
  "MemorySummary": {...},
  "Processors": {...},
  "EthernetInterfaces": {...},
  "LogServices": {...},
  "HostedServices": {...},
  "Links": {...},
  "Actions": {...}
  }
}

Note: For brevity, I removed the value information for many of the properties. You can see the full output on Mockable.IO.

Example 3: Easily identify hosting and containment relationships

I mentioned in the previous example that a StorageSystem can host one or more StorageServices. A StorageService is the entity that presents StoragePools, Volumes, and FileSystems. In the information returned for the SIMPLE StorageSystem, look for HostedServices property.

 "HostedServices": {
    "StorageServices": [
        {"@odata.id":"/redfish/v1/StorageServices/Simple1"}
    ]
}

The @odata.id value points to a StorageService names SIMPLE1. StorageServices entities are listed directly under the ServiceRoot. The underlying Swordfish implementation is responsible for returning this information as part of the GET API. With this @odata.id value, you can construct the following URI:

 https://swordfish.mockable.io/redfish/v1/StorageServices/Simple1

The API call will return all the information for the SIMPLE1 StorageService hosted on SIMPLE StorageSystem.

 {
  "@Redfish.Copyright": "Copyright 2015-2016 SNIA. All rights reserved.",
  "@odata.context": "/redfish/v1/$metadata#StorageService.StorageService",
  "@odata.id": "/redfish/v1/StorageServices/Simple1",
  "@odata.type": "#StorageService.1.0.0.StorageService",
  "Id": "Simple1",
  "Name": "My Simple Storage Service",
  "Description": "This simple system does not support service levels when provisioning storage",
  "Status": {
    "State": "Enabled",
    "Health": "OK"
  },
  "StorageGroups": {...},
  "InitiatorEndpointGroups": {...},
  "TargetEndpointGroups": {...},
  "Endpoints": {...},
  "StoragePools": {...},
  "Volumes": {...},
  "Drives": {...},
  "Links": {...},
  "Oem": {}
}

Note: For brevity, I removed the value information for many of the properties. You can see the full output on Mockable.IO.

A StorageSystem can have multiple StorageServices with different capabilities. For example, a StorageSystem can have a present SAN services seperate from NAS. In a multi-tenant environment, the StorageSystem can have separate StorageServices for each tenant. StorageServices is one way that Redfish/Swordfish organizes logical entities separate from physical/virtual entities like StorageSystem.

If you want to see all the StorageServices managed by the Swordfish endpoint use the following API:

 https://swordfish.mockable.io/redfish/v1/StorageServices

Example 4: Quickly drill down to get specific information about a resource

Now that you know how to get StorageSystem and StorageService information, lets say you want to build a simple report that tracks the storage capacity of a storage pool over time. Looking at the information returns for SIMPLE1 StorageService you see the StoragePool property.

 "StoragePools": {
    "@odata.id": "/redfish/v1/StorageServices/Simple1/StoragePools"
}

Using the @odata.id you can construct get the list of StoragePools contained by SIMPLE1 StorageService:

 GET https://swordfish.mockable.io/redfish/v1/StorageServices/Simple1/StoragePools

Response:

 {
  "@Redfish.Copyright": "Copyright 2015-2016 SNIA. All rights reserved.",
  "@odata.context": "/redfish/v1/$metadata#StoragePoolsCollection.StoragePoolsCollection",
  "@odata.id": "/redfish/v1/StorageServices/Simple1/StoragePools",
  "@odata.type": "#StoragePoolsCollection_1_0_0.StoragePoolsCollection",
  "Name": "StoragePools",
  "Members@odata.count": 1,
  "Members": [
    {
      "@odata.id": "/redfish/v1/StorageServices/Simple1/StoragePools/BasePool"
    }
  ]
}

Using the @odata.id for the listed member, get detailed capacity information about the BASEPOOL StoragePool. Specifically, you can use Capacity/Data property to get AllocatedBytes and ConsumedBytes.

 GET https://swordfish.mockable.io/redfish/v1/StorageServices/Simple1/StoragePools

Response:

 {
  "@Redfish.Copyright": "Copyright 2015-2016 SNIA. All rights reserved.",
  "@odata.context": "/redfish/v1/$metadata#StoragePool.StoragePool",
  "@odata.id": "/redfish/v1/StorageServices/Simple1/StoragePools/BasePool",
  "@odata.type": "#StoragePool_1_0_0.StoragePool",
  "Id": "BasePool",
  "Name": "BasePool",
  "Description": "Base storage pool",
  "BlockSizeBytes": 512,
  "Capacity": {
    "Data": {
      "AllocatedBytes": 10995116277760,
      "ConsumedBytes": 3760197652418
    }
  },
  "Status": {
    "State": "Enabled",
    "Health": "OK",
    "HealthRollUp": "Degraded"
  },
  "Links": {
    "AllocatedPools": [],
    "AllocatedVolumes": [
      {
        "@odata.id": "/redfish/v1/StorageServices/Simple1/Volumes/1"
      }
    ],
    "SourceDevices": [],
    "SourcePools": [],
    "SourceVolumes": [],
    "SupportedClassesOfService": []
  }
}

If you call the GET API again you will see Capacity/Data/ConsumedBytes changes. Since this is only a mockup, the value is changes dynamically. Using a simple PowerShell script, you can out the ConsumedBytes value every minute for an hour and plot the following graph in Excel:

 $ConsumedGB = @()
foreach ($i in 1..60)
{
    $basepool = Invoke-RestMethod -Uri "https://swordfish.mockable.io/redfish/v1/StorageServices/Simple1/StoragePools/BasePool" 
    $capacity = New-Object System.Object
    $capacity | Add-Member -type NoteProperty -name Time -value $i
    $capacity | Add-Member -type NoteProperty -name ConsumedGB -value ($basepool.Capacity.Data.ConsumedBytes/1000000000)
    $ConsumedGB += $capacity
    Start-Sleep -Seconds 60
}
$ConsumedGB | Select Time, ConsumedGB | Export-Csv -Path $env:temp\consumedgb.csv -Encoding ascii -NoTypeInformation

ConsumedBytesinTBGraph

Example 5: You can use PowerShell objects to navigate Swordfish resources

Using PowerShell, you can use the Invoke-RestMethod cmdlet to retrieve all the information in the examples above.

I went ahead and created a simple PowerShell script that performs deep discovery of Chassis, StorageSystems, and StorageServices from the ServiceRoot and stores the information into a single JSON document.

SwordfishDemoScript.ps1

Download the script above and open it in PowerShell ISE (its easier to demo using this interface). Run the script once to register the functions contained in the script. Un-comment the two lines under #MinimalExample, select the lines, and execute that selection.

MinOut

The variable $discoveryinfo contains all information about the StorageSystems and StorageServices. You can easily get to ConsumedBytes (Intellisense works here):

ConsumedBytes

You can also throw the output to JSONViewer. Un-comment the three lines under #VerboseExample, select the lines, and execute that selection.

VerOut

You can take the output in the Notepad window and paste it into JSONViewer. JSONViewer is available at https://jsonviewer.stack.hu.

Expand StorageServices/Simple1/StoragePools/BasePool/Capacity/Data/ConsumedBytes

JSONViewerConsumedBytes

These are only 5 examples showcasing the new Swordfish API. Swordfish v1.0 will release with its own documentation so you can dig deeper.

We hope to see you at Microsoft Ignite in Atlanta this year. If you are interested in participating the end-user meet-up that SNIA is hosting on September 27, please fill out the survey to RSVP and guarantee your spot.

Resources

Description Link
Swordfish Homepage https://www.snia.org/swordfish
Swordfish v0.9 Work-in-Progress https://www.snia.org/publicreview#swordfish
Redfish Homepage https://www.dmtf.org/standards/redfish
Swordfish Presentations Live stream of the general sessions on Tuesday and Wednesday morninghttps://snia.us7.list-manage.com/track/click?u=b421c01fc38e58ca8bdccc0be&id=e824ca6001&e=908692e382Introduction to Swordfish: Scalable Storage Management - SNIAhttps://www.snia.org/sites/default/files/DSI/2016/presentations/stor_mngmnt/RichelleAhlvers_Introduction_to_Swordfish.pdf
JSONViewer https://jsonviewer.stack.hu
Swordish APIs Mockups https://swordfish.mockable.io/redfish/v1
SNIA End-user Meet-up @ Ignite https://www.surveymonkey.com/r/sniaignite16