Showcasing Camel with a simple IRC bot

Who or what is Camel, anyway?

Camel is... the new black a horse designed by a committee a horse called the ship of the desert

Ah, naming. The bane of Computer Science. In software-land, Camel is an enterprise integration library from Apache.

I hear you asking what, exactly, that really means, and you wouldn’t be alone. My take: If you find yourself writing code to make your (JVM-based) program talk to another program, you’re doing it wrong. Camel has a plethora of components that let you integrate disparate systems via routes. A Camel Route is remindful of a sequence of Unix pipes: it lets you pull data or receive events from a source, apply transformations, and send data to a sink. Most “components” work as either, but you’d usually set different options depending on the usage. You can use routes to implement many Enterprise Integration Patterns, which means that the whole project is firmly rooted in real world usage. A quick browse of the patterns page should leave you with a solid feeling of usefulness.

Moving on to something less abstract, this post shows how to use Camel to create a simple IRC bot that stores messages in a MongoDB database. The real point of the blog is to demonstrate Camel’s capabilities, but the code does run and you can use it to preserve your favourite IRC channel activity for posterity[1. If you’re not a IRC user, the inability to read the activity that a channel might have had prior to your joining it is frequently cited as an inconvenience of the service.].

Much goodness

Butt kicking - for goodness!I tend to be too cynical, so with the help of my goodness editor here, I’ll make sure to point out the good stuff early on. Featured in this post: Apache Camel (integration), Heroku (cloud hosting for apps), Mongolab (cloud storage), Scala (great language for the JVM), and Play (an MVC web framework for the JVM in the spirit of Rails).

 

Using Camel

Camel runs in your JVM of choice. You’ll need to import a lot of jars, hopefully with the help of an automated dependency management tool. The project itself documents dependencies with Maven and it is carefully modularized so that you only need to bring in the specific packages you want. Camel works well with Java, obviously, but it also works nicely with Scala. There is a Scala version of the route definition DSL and there is also the Akka-Camel module that lets you integrate Akka actors with Camel routes. I am using this module for a middleware product and can only recommend it.

For the purpose of this post I created a small Play application[2. There we are again with the naming.]. I am not really using Play for anything other than its beautiful Json library; in all honesty the usage of Play is just the beginning of a series of preposterous abuses of modern technology and services, all for the purpose of running a lousy IRC bot. I wanted a cloud-based app host to run my bot, and Play just so happens to be supported by Heroku. I also need somewhere to store the messages; since Heroku only provides an ephemeral file system[3. Quite reasonably when you think about it, but it does surprise developers a lot.], I went with MongoDB, because I can. Also, there exists Mongolab[4. Both Heroku and Mongolab have free usage tiers.].

But I digress. Camel requires launching a Camel Context where your routes will live. The context and its routes is defined using the Camel DSL, either in source code or in Spring XML. In accordance to its Java heritage, you will find many ways to achieve the same goal with Camel; it is all about choices, and depending on your situation you will want to define some things in code, others in XML, or even plugging in your own Camel components (like an aggregation strategy for the immensely powerful Aggregator component[5. An upcoming post will discuss creating an aggregation strategy to continuously stream output to a sink.]).

Ircast

Let’s go over the details taking the bot as a guide / example. You can get the code at Github, which is mostly a basic Play application built with Scala; here are the highlights:

build.sbt

SBT, the Scala Simple Build Tool, handles dependencies (using Ivy) and builds the application. In this case:

[scala]
name := "ircast"
version := "1.0-SNAPSHOT"

libraryDependencies ++= Seq(
cache,
"org.apache.camel" % "camel-scala" % "2.12.2",
"org.apache.camel" % "camel-spring" % "2.12.2",
"org.apache.camel" % "camel-irc" % "2.12.2",
"org.apache.camel" % "camel-mongodb" % "2.12.2",
"org.springframework" % "spring-beans" % "3.+",
"org.springframework" % "spring-context" % "3.+"
)

play.Project.playScalaSettings
[/scala]

As you can see the code above is declaring dependencies on a series of modules. These dependencies will translate to .jar downloads from Maven Central to your local working copy or, thanks to Heroku’s Play support, to your Heroku deployment[6. I firmly believe that you should develop software with a proper automated build system, so being able to push only the code to Heroku, who will in turn repeat your local build process (download jars, build classes, etc.), is very satisfying for me.]. The Play runtime handles downloading dependencies and linking to them in runtime quite transparently. The runtime even supports reloading classes on the fly to give you a quicker feedback loop a la Rails.

camel-context.xml

I am using the Spring-based XML DSL for this example[7. Yes, XML sucks, and XSD with namespaces only makes it worse, but there are advantages to externalize parts of your application.]. This is my route:

[xml]
<camelContext id="camel" trace="false" xmlns="http://camel.apache.org/schema/spring" streamCache="true">

<route id="consume">

<from uri="irc:ircast@chat.freenode.net?channels=#scala-mvd&amp;nickname=ircast&amp;onReply=false&amp;onNick=false&amp;onQuit=false&amp;onJoin=false&amp;onKick=false&amp;onMode=false&amp;onPart=false&amp;onTopic=false"/>

<to uri="log:logger.app?showAll=true"/>

<bean ref="processor" method="setJson"/>

<to uri="mongodb:ircastDb?database=ircast&amp;collection=scala-mvd&amp;operation=insert"/>

</route>

</camelContext>
[/xml]

Let’s go over the route in detail; there is another block of XML below completing some missing parts (like the “processor” bean above).

from uri=”irc:ircast (…)

Tells Camel to monitor and react to messages in a IRC channel, in this case #scala-mvd at freenode.
This means that when someone types “Howdy!” in the channel an exchange will flow down this route, which is an instance of Exchange that has

  1. properties
  2. an “in” message
  3. possibly an “out” message (if a reply is expected)

“In” and “Out” messages consist of headers (pieces of metadata identified by names) and a body. It is up to the component to define what goes where, and indeed different components have very different headers, though usually the body has what you’d expect. In the case of the IRC component, for example, the body is the text of the message, and the headers include “irc.user.nick” (the user who sent the message).

<to uri=”log:logger.app?showAll=true”/>

Just to log the exchange, including all properties and headers. Helpful to know what’s going on. The “logger.app” part there is the name of the logger. Play seems to like loggers whose names are prefixed with “logger”. Just to make sure it’s a logger and not some other kind of logger. Loggers.[8. Sometimes programming for the JVM feels as it’s logging frameworks all the way down.]

<bean ref=”processor” method=”setJson”/>

I need to turn the body of the In message of the exchange into a Json object to save the information in MongoDB (see the next step). This line calls the setJson method of a regular Java class (created with Spring):

[scala]
import org.apache.camel._
import play.api._
import play.api.Play.current
import play.api.libs.json._

class MsgProcessor {
def setJson(exchange: Exchange) {
val jsonObj = Json.toJson(
Map(
"data" -> exchange.getIn.getBody.toString,
"user" -> exchange.getIn.getHeader("irc.user.nick").toString,
"date" -> exchange.getProperty(Exchange.CREATED_TIMESTAMP).toString
)
)
val newBody = Json.stringify(jsonObj)
Logger.info(s"Processing $newBody")
exchange.getIn.setBody(newBody)
}
}
[/scala]

I apologize for mutating data like that. The code above turns the original “Howdy!” into a simple Json map adding a timestamp (taken from the Exchange properties) and the name of the user that sent the message (taken from the In message headers).

to mongodb:ircastDb

Camel doing its magic. The Camel MongoDB component will save the body of the In message as long as it is a DBObject or a Json object. Since the previous component changed the original String into a Json map, the message it’s ready to go, and will get saved in MongoDB, just like that.

mongolab_doc

Other details

The rest of camel-context.xml

Missing above, these important bits create the bean “processor” that creates the Json object for insertion in MongoDB, and the MongoDB client to connect to the database at Mongolab.

[xml]

<bean id="routes" class="MsgProcessor"/>

<bean id="mongoUri" class="com.mongodb.MongoClientURI">
<constructor-arg index="0" value="mongodb://ircast:password@server.mongolab.com:port/ircast"/>
</bean>

<bean id="ircastDb" class="com.mongodb.MongoClient">
<constructor-arg index="0" ref="mongoUri"/>
</bean>

[/xml]

Camel initialization

This class hooks into the Play framework to read the camel-context.xml file when the application starts:

[scala]

import play.api._
import org.springframework.context.support.ClassPathXmlApplicationContext

object Global extends GlobalSettings {

override def onStart(app: Application) {
Logger.info("Application has started")
initCamel
}

def initCamel() {
// Start Camel
new ClassPathXmlApplicationContext("camel-context.xml")
}
}

[/scala]

Conclusion

Remember: if you need some systems to work together, consider Camel. Your integration needs can’t be crazier than pushing IRC messages to MongoDB.

Posted in Uncategorized

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>