General

What does Seiso mean?

Seiso (清掃) is a Japanese word meaning "clean, shiny, tidy". The idea is that we can enable better automation by organizing and maintaining our data in a clean, tidy and maybe even shiny way.

Where can I get the Seiso binary?

We haven't yet put a binary up since there are a few things we want to fix before doing that. You can expect a binary in January 2015. Until then please clone the repo and build from source.

How do I deploy Seiso to a production environment?

We don't recommend this yet, as Seiso is currently alpha software. As the project matures we will provide an appropriate init script, Chef cookbook and documentation to support this.


Importing data

How do I get data into Seiso?

Use the REST API. We have a Seiso API v1 Tutorial as well as a Seiso API v1 Reference.

See also the data integration FAQ below.

Can I enter/update/delete data through the user interface?

No. Seiso isn't intended to be a master for the data. Instead other systems master the data, and automation engineers create integrations—whether batch, real-time or a combination of the two—to keep Seiso in sync with the masters.

Here's why. Suppose that you're pulling data into Seiso from Active Directory, Chef server and your NetScaler load balancers. If we were to let users update such data through the Seiso UI, we would have to have a way to push this back to the source systems. While it could be done, it many cases it would be too complicated to resolve conflicts between the various entry points into the source systems. (Think of having to manage conflicts between Seiso and the Chef Ohai plugin, or between Seiso and data coming out of a version control system.) So by design the Seiso UI is read-only.


Data schema

Seiso doesn't support a data field I need. Can you add it?

Create a GitHub issue describing your idea. We generally prefer fields and item types that help with data organization and integration. For example:

  • Service groups organize services, and status types organize health and rotation statuses; these are examples of organizational item types.
  • Services, nodes, machines and IP addresses integrate deployment and operational data.

Sometimes users request fairly specialized and/or fine-grained fields and types, presumably because they'd like Seiso to be a one-stop shop for all of their automation data. Seiso isn't really designed for this. To offer a concrete example, when we sync Seiso with Chef server, we don't pull in every bit of Ohai node data, because it's too huge, and most of it isn't broadly useful. We try to focus on the key integration data.

We plan to support dynamic, user-defined properties and tags at some point, which should help in cases where the proposed fields are too specialized or fine-grained to be part of the data schema.

My team doesn't know JSON. Can you add XML support?

While it would be technically feasible to support XML, we've made a conscious decision not to. JSON is more concise and works more seamlessly with JavaScript clients, like our AngularJS web UI. Supporting XML would roughly double the number of endpoints to test and support. We'd prefer to dedicate our efforts to other features.


Integration

Do Seiso clients have to be written in a specific language?

Nope. Seiso is a REST API, so you can use whatever client language and/or platform you like. This is by design, as we intend Seiso to support large technology organizations, and such organizations often use a variety of technology platforms.

At Expedia we have Seiso clients written in Java, C#, JavaScript, Ruby, PowerShell, bash and Perl.

How do I integrate Seiso with other services?

Seiso supports two complementary service integration approaches:

  • REST API: Your services can call endpoints on the Seiso RESTful web service API. In general these are standard verbs (GET, POST, PUT and DELETE) on the various items that Seiso manages.
  • RabbitMQ messaging: Seiso pushes AMQP messages into RabbitMQ following data changes. Your services can listen for such messages and respond in real time.

Specifically, how should I integrate with data sources?

This is a large topic, and the answer varies depending on the nature of the data in question. Let's briefly cover some typical cases. We have guides covering these topics in more detail.

Source system has semi-dynamic data. Often there are already systems in place that manage the data you want to pull in, but the data isn't particularly dynamic. You might have person data in Active Directory, for instance. Here one option is to create batch jobs that sync the data on a regular period.

Source system has highly dynamic data. Sometimes however you do need something more real-time. Launching and terminating cloud instances are one example, and endpoint rotation state in a load balancer are another. There are at least two common approaches here:

  • Wrap access to the dynamic source, and then have the wrapper invoke Seiso's REST API when it generates relevant changes. Scripts that provision machines or deploy software might tell Seiso after they successfully complete.
  • Create a listener—or high-frequency poller if listening isn't an option—that can invoke Seiso's REST API when events come in. You might listen for SNMP traps coming out of the load balancer, for example, and then call Seiso.

Multiple source systems. Sometimes there are multiple source systems for the sort of data you care about. Maybe one part of the organization maintains its machine data in Chef, whereas another uses a custom database. Here you can just create separate jobs to import the data into Seiso for a consolidated, normalized view.

No existing source system. Sometimes there isn't an existing system for the data you want to put in Seiso. Maybe you don't have master list of services (sometimes called a "service catalog") or a master list of environments. At Expedia we've found it effective to put this kind of data in files under version control, and then create continuous integration jobs (e.g., Bamboo or Jenkins jobs) that respond to changes by pushing the data into Seiso via the REST API. (We will likely open source these integration scripts at some future date.)

These are just some common approaches. No doubt you will think of others. Seiso is pretty flexible with respect to how you do this.

Seiso says my endpoint is in rotation, but it's really not. What's up?

Seiso is just a repository for the rotation status data (and all other automation data), meaning that it just reports what your integration code most recently told it. So check your integration approach and/or code.

For what it's worth, we've seen this kind of behavior in a variety of contexts:

  • It arises in polling situations where the integration code uses a polling interval that's too large. We recommend using event-driven updates where feasible.
  • It can arise when dual-sourcing rotation data from primary/failover devices that disagree with one another.
  • It can happen if you have a race between a real-time update mechanism and a periodic batch mechanism. The race works like this: the batch job grabs a bunch of rotation status data, the rotation status changes, the real-time mechanism updates Seiso, and then the batch job updates Seiso with stale data, undoing the real-time update.

Obviously there are lots of ways to put wrong data into Seiso. If you think that you've found a bug with Seiso itself, by all means create an issue in GitHub .


Contributing

How can I contribute?

There are multiple ways to contribute:

  • Use Seiso and get the word out. If you like Seiso, we'd your help in getting the word out through Twitter and other social media.
  • Request enhancements and report issues. See below.
  • Create blog posts, tutorials and documentation. Did you figure out a cool way to integrate Seiso with your issue tracking system? Write a blog post or a tutorial and let us know about it so we can help socialize it.
  • Contribute integration scripts. Even better, if your integration is something that others might use directly, put it on GitHub.
  • Contribute code. We welcome code contributions. Start by creating an issue in GitHub so we can discuss your idea. After we work through the idea, write the code (please include unit tests) and create a pull request. We'll work with you to get it merged.

How can I request enhancements or report bugs?

Just go to our GitHub site and create an issue there.