Project Iris

For a brief overview of Project Iris, read Tyler Treat’s blog post.

Why Iris?

  • cluster is based on namespace (akin to Kubernetes pod), not IP address (no need for service discovery tools like etcd, consul)
  • minimal API (client bindings in Go, Scala, Erlang)
  • built-in security
  • works automagically with private subnet
  • open source
  • written in Go
  • mix and match using four models of messaging
  • I prefer Debian 7 but this also works with Windows

For this post, I will just discuss the publish/subscribe aspect.

Starting the Iris relay is just one command (I just renamed the binary to iris for brevity). I have used version 0.3.2

iris -dev (for development)

iris -net clustername -rsa path/of/RSA private key (for production)

Below is publish and subscribe clients in Go (GitHub code here and here).
This publisher generates random 320-character messages up to a number of your choosing (default is 10,000).

package main(
import (
    "flag"
    "fmt"
    "gopkg.in/project-iris/iris-go.v1"
    "log"
    "math/rand"
    "time"
)
var letters = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!@#$%^&*()1234567890")
var numbPtr = flag.Int("msg", 10000, "number of messages (default: 10000)")
func randSeq(n int) string {
    b := make([]rune, n)
    for i := range b {
        b[i] = letters[rand.Intn(len(letters))]
    }
    return string(b)
}
func main() {
    flag.Parse()
    conn, err := iris.Connect(55555)
    if err != nil {
        log.Fatalf("failed to connect to the Iris relay: %v.", err)
    } else {
        log.Println("Connected to port 55555")
    }
    start := time.Now()
    for i := 1; i <= *numbPtr; i++ {
        conn.Publish("test", []byte(randSeq(320)))
    }
    elapsed := time.Since(start)
    log.Printf("Time took %s", elapsed)
    defer conn.Close()
    fmt.Scanln()
}

============================================================================
//subscribe.go
package main
import (
    "flag"
    "fmt"
    "gopkg.in/project-iris/iris-go.v1"
    "log"
    "runtime"
    "sync/atomic"
    "time"
)
type topicEvent struct{}
var start = time.Now()
var ops uint64 = 0
var numbPtr = flag.Int("msg", 10000, "number of messages (default: 10000)")
func (t topicEvent) HandleEvent(event []byte) {
    log.Println(string(event))
    atomic.AddUint64(&ops, 1)
    if ops == uint64(*numbPtr) {
        elapsed := time.Since(start)
        log.Printf("Time took %s", elapsed)
    }
}
func main() {
    flag.Parse()
    conn, err := iris.Connect(55555)
    if err != nil {
        log.Fatalf("failed to connect to the Iris relay: %v.", err)
    } else {
        log.Println("Connected to port 55555")
    }
    var topicHandler = new(topicEvent)
    conn.Subscribe("test", topicHandler, &iris.TopicLimits{
        EventThreads: runtime.NumCPU(),
        EventMemory:  64 * 1024 * 1024,
    })
    defer conn.Close()
    fmt.Scanln()
}

Now here comes the testing stage. We need to run the subscribe client first so that the publish client would not lost messages. Running without the msg flag is fine. The subscribe clients would get all the default 10,000 messages but depending on your computer, it would run to a race condition.

This race condition is not a bug per se of Iris but it means, you may have reached the memory or CPU limits depending on the task of your subscribe client.

Regardless of message queue software you are using, idempotence is the responsibility of the application developer.

Idempotence (a+a=a), duplication of message or data doesn’t matter

Now, the fun part is when you run two instances of subscribe clients, and the race condition disappears.

Granted that Iris is not a full-fledged message queue like others, it is a DIY message queue solution. Depending on the complexity of your tasks, you may have at least two instances of subscribe clients preferably on different physical nodes, which in turn, will be responsible for routing the data to the actual worker clients.

The project is young but it looks promising. Iris is a model of refactoring  a system into an engine (the Iris relay which is doing the heavy lifting) and into an API (it is minimal and user-friendly).

Other Iris examples in Go can be found at my GitHub repo: https://github.com/ibmendoza/go-examples

If you are a Go, Scala or Erlang developer, you owe it to yourself to test drive this amazing project. It is a waste of time to ignore this project.

Advertisements

Subjectivity aside, leave a reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s