FIXED: Amazon EC2 vulnerable to UDP flood attacks

Unfortunate events surrounding the DDoS attack against BitBucket kicked-off heated discussions about the nature of this vulnerability. Where Amazon officially acknowledged this to be a single isolated incident, many others started asking questions why did it happen in first place?
- Was BitBucket’s security group configuration set to block UDP traffic?
- How come they haven’t got better visibility of the on-going attack?
- Is this really Amazon’s fault?
Both personal and professional interest led me to find out more. Having designed series of tests how to replicate this scenario, started first instance and set up the target environment.
instance : c1.medium (us-east-1d)
EBS volume : 200 GB attached to (/dev/sdf)
monitoring : vmstat, netstat, iptraf, Amazon CloudWatch
security group : allowed SSH only (port 22/TCP)
UDP flood set up to be generated from the second instance (c1.medium) using simple Perl script, managing to generate whopping traffic of 650mbit per second (according to iptraf) using 1KB packets to random ports on the target IP.
<b>Test 1. Let it run</b> has been successful in a way there was no visibility on target machine. Still surprised by the traffic level generated on the source box, I’ve pointed the UDP flood to another machine – with security group allowing UDP traffic (ports 0 – 65535) – to check if the network traffic is able to reach another box. And it was. Not only from the same availability zone, but even from the different ones (tested us-east-1c and us-east-1b).
<b>Test 2. consisted of formatting the prepared EBS, 5 samples for both scenario with and without UDP flood.
Average Time
no traffic 1m15s
UDP flood 2m54s
During the test there were only moderate increase in IO waits (somewhere between 2 – 4%)
<b>Test 3. Bonnie++ performance test of the EBS volume. Running with no incoming traffic, it took around 8 minutes to produce quite reasonable report. Having switched on the UDP flood I’ve repeated the same tests and my expectation was to see some results in similar time. Fifteen minutes later and bonnie still haven’t even finished third step (rewriting). Another 10 minutes without any significant progress pointed me to do some research what’s going on. The box wasn’t performing virtually any IO operations, and time spent waiting for IO topped 100% every second reading (1s delay). Bingo!
To verify if the problem is really caused by incoming UDP flood, I’ve stopped the traffic for a brief interval (around 7 seconds) and monitored using vmstat:
procs ———–memory———- —swap– —–io—- -system– —-cpu—-
0 1 893480 11272 3112 1699240 0 0 0 0 10 11 0 0 66 34
0 1 893480 11272 3112 1699240 0 0 0 0 9 6 0 0 0 100
0 1 893480 11272 3112 1699240 0 0 0 0 10 9 0 0 67 33
0 1 893480 11272 3112 1699240 0 0 0 0 11 8 0 0 0 100
0 1 893480 8824 3100 1700052 0 0 23808 24864 962 697 0 1 68 31
0 1 893480 12284 3084 1697988 0 0 16384 16576 711 424 0 2 4 93
0 1 893480 9020 3084 1700088 0 0 20480 20720 817 563 0 1 68 31
0 1 893480 10432 3072 1700192 0 0 20864 20720 907 612 0 4 5 90
0 1 893480 10976 3040 1699724 0 0 15620 12432 588 423 0 1 68 31
0 1 893480 10872 3044 1698556 0 0 12676 16576 600 350 0 2 2 96
0 1 893480 10328 3024 1700676 0 0 19976 16576 761 535 0 1 68 31
0 1 893480 12408 3004 1698096 0 0 8708 12432 457 254 0 1 4 95
0 1 893480 12408 3004 1698096 0 0 0 0 9 7 0 0 67 33
0 1 893480 11636 3004 1699120 0 0 1024 0 38 38 0 0 0 100
0 1 893480 10548 3004 1700420 0 0 1280 0 47 45 0 0 66 33
0 1 893480 10188 3004 1700756 0 0 3584 4144 195 110 0 0 0 100
0 1 893480 10120 2992 1697968 0 0 6404 8288 256 205 0 0 67 33
0 1 893480 12468 2992 1696864 0 0 8064 8288 343 250 0 0 2 98
0 1 893480 11720 2972 1696984 0 0 12420 12432 495 333 0 0 67 32
0 1 893480 10136 2976 1700800 0 0 6916 4144 321 190 0 0 0 100
0 1 893480 11972 2956 1698820 0 0 4096 4144 161 117 0 0 67 33
0 1 893480 11364 2960 1699480 0 0 3844 4144 200 126 0 0 1 99
0 1 893480 11432 2960 1699480 0 0 2944 4144 160 91 0 0 66 34
0 1 893480 11156 2960 1699820 0 0 256 0 18 12 0 0 0 100
0 1 893480 10884 2960 1700020 0 0 256 0 17 17 0 0 66 34
0 1 893480 10856 2960 1700076 0 0 0 0 9 8 0 0 0 100
0 1 893480 10856 2960 1700076 0 0 0 0 9 9 0 0 67 33
As you can see on line 5 the IO traffic resumed, roughly correlating to the time incoming traffic stopped. Seven seconds later with the UDP traffic back on the box tried to keep up for another quarter of minute before giving it up. Best time to check CloudWatch:
<cloud watch image>
Nothing! Based on my notes the first bonnie run occured at 10:40, switched on the UDP flood at 10:50, and started second bonnie run at 10:52. My patience ran out before 11:30 where there’s small peak caused by interactive iptraf session.
At this point there were no reasons to continue testing. All IO operations to/from EBS volume seemed to be blocked by UDP traffic generated by a single instance!
Conclusion
BitBucket guys had every reason to be angry. Blocking UDP in the security group configuration only hides the problem. Contraindicating the Jesper Nøhr statement, during this experiment there were no peaks visible using paid monitoring service – Amazon CloudWatch (see above). Which was probably the amount of information available for AWS 1st line of support.
This corresponds to the ‘black box’ described by Jesper. Looking back on the results it’s obvious that
- on-demand network capacity backfired in this case
- security group configuration is most likely applied on the host system
- host architecture seems to be sharing same network interface(s) for actual network traffic as well as network traffic to/from EBS instances. Even though instances got only a single network interface, I would expect this separation to be implemented on the host system. Segregation of the network traffic is one of the first lesson learned in high-exposed clustered environment.
- a week after the attack and there isn’t any fix in place. Hello, Amazon?!?!
To be fair, it’s been the first incident of such a magnitude. Let’s hope Amazon AWS team will come up with the architecture fix before somebody use the vulnerability in much wider and devastating attack. In mean time, the only workaround we can apply is to hide our instances as much as we can. Load-balancers and proxies in front of the worker instances should be enough, as long as you don’t share the same host machine.
Have a good weekend and good luck protecting your instance’s IPs!
PS: who had the same dark thought as I just had? What about S3?

UPDATE 2009-10-12: I’m happy to let you know this post is not longer relevant. Amazon AWS team successfully deployed the fix and the scenario used to simulate Denial of Service attack using UDP flood isn’t applicable anymore. All that in less than 24 hours after publishing the link on Twitter. Good job!

Original post follows.

Read More »

Open Source as a commodification driver

If you play with fire, you’ll get burned. Exactly same happened to me in the post Cloud Perspectives, in which I had wandered off my usual comfort zones and with proper amount of clumsiness stumbled upon the topic I haven’t been able to describe properly. As it was too late to delete it, I am bound to explain myself properly.
We we all agree the technology is driven by the innovation and the value added to its users. At the beginning of every progression there’s an innovation that demonstrates potential economic value assigned to something previously not recognized or possible. In most areas the innovation (IP) and it’s strategic value plays important part in the establishing or changing market place. Once the strategic value drops significantly, either by increased competition or by mass adoption, commodization is taking progress. From technology perspective it usually happens by:
- standardization
- adoption growth not manageable by a single or limited number of entities
- transfer of IP to 3rd parties
- containment of innovation itself in the mainstream know-how
In the process of commodification the competitive rivalry of the market will sooner or later face the loss of added value and therefore diminishing strategic value of particular innovation and diversity of competitors – finishing the process of commodification by the containment of original idea in the general know-how. Which will created environment best described as a pure competition (see Porter’s Five Force Analysis [http://en.wikipedia.org/wiki/Porter_5_forces_analysis]).
Now let’s take the Open Source into the equation. The movement itself is already recognized by doing all of this – removing technological barriers for adoption, providing a platform for innovation, reducing the cost of ownership compared to the proprietary technologies and establishing new environment for de-facto standardization of selected projects. Which, on its own, should give us perfect strategic way of achieving technology commodization.
The question remains though. Right now Open Source has been reduced to handful of products in order to simplify it and come up with the new business model. But reality is much more complex. Open source has been one of the major enablers of what we’re calling cloud computing, changing technological landscape as we knew it. As noted by Roman in his original post, the cloud environment comes with different layer of interaction. The technological challenges we’ve faced before are already recognized and therefore commodified. Now it’s a question what new projects (or frameworks) swe’ll have to solve the new challenges? Or better – as service nature of cloud computing changed the impact of open source is there a way how to project an open source ideas into them as well?

If you play with fire, you’ll get burned. Exactly same happened to me in the post Cloud Perspectives, in which I had wandered off my usual comfort zones and with proper amount of clumsiness stumbled upon the topic I haven’t been able to describe properly. As it was too late to delete it, so I am bound to explain myself properly.

We we all agree the technology is driven by the innovation and the value added to its users. At the beginning of every progression there’s an innovation that demonstrates potential economic value assigned to something previously not recognized or possible. In most areas the innovation (IP) and it’s strategic value plays important part in the establishing or changing market place. Once the strategic value drops significantly, either by increased competition or by mass adoption, commodization is taking progress. From technology perspective it usually happens by:

  • standardization
  • adoption growth not manageable by a single or limited number of entities
  • transfer of IP to 3rd parties
  • containment of innovation itself in the mainstream know-how

In the process of commodification the competitive rivalry of the market will sooner or later face the loss of added value and therefore diminishing strategic value of particular innovation and diversity of competitors – finishing the process of commodification by the containment of original idea in the general know-how. Which will created environment best described as a pure competition (see Porter’s Five Force Analysis).

Now let’s take the Open Source into the equation. The movement itself is already recognized by doing all of this – removing technological barriers for adoption, providing a platform for innovation, reducing the cost of ownership compared to the proprietary technologies and establishing new environment for de-facto standardization of selected projects. Which, on its own, should give us perfect strategic way of achieving technology commodization.

The question remains though. Right now Open Source has been reduced to handful of products in order to simplify it and come up with the new business model. But reality is much more complex. Open source has been one of the major enablers of what we’re calling cloud computing, changing technological landscape as we knew it. As noted by Roman in his original post, the cloud environment comes with a different layer of interaction. The technological challenges we’ve faced before are already recognized and therefore commodified. Now it’s a question what new projects (or frameworks) we’ll have to solve the new challenges? Or better – as service nature of cloud computing changed the impact of open source, is there any way how to project an open source ideas into the service provisioning as well?

Cloud Perspectives

In his latest blog post, Roman Stanek contemplated about the disruptive effect of cloud computing. We already experienced its power in relation to the boom of social networking and as he notes it is undoubtedly going to disrupt the business model behind commercial software. Now, Roman asks how it will affect the open source movement?

For me the answer to this question lies in the position of the cloud computing itself. As a technology it’s nothing more than an evolutionary step. To understand my view let’s make a step back and have a look into the history and see what preceded our latest source of buzz.

First here was a network connectivity. Back in 1995 I tried to build my very first ISP. Daring I hear you saying? It certainly was. Some tried to fight it, some opposed it, most of us were amazed by the possibilities (however limited at the time) and the others were just oblivious. Connectivity was scarce, expensive and limited. As adoption progressed all these problems started to disappear. By the end of the 90s connectivity wasn’t any longer a privilege but a commodity. Bought, resold, aggregated, and extended. Would you even considering building own ISP today? I doubt so [1].

Next on the line was hosting. Everybody who missed the ISP game, and even ISPs themselves, entered a big chase for the ultimate hosting package. Story is the same as with connectivity. Originally it was scarce, expensive and limited. And its fate is similar as well. Hosting as a service has been commodified. You know the story.

As our needs increased, the next step was to focus on data centres. Transit connectivity and mass demand for hosting services increased pressure to provide huge data centres. Server provisioning become important. Numbers changed, from dozens to hundreds, from hundreds to thousands. You can probably guess what I’m going to say: it was scarce, expensive and limited. Physical kit just wasn’t enough. Virtualization matured and brought as where we are know. Into the cloud age.

Open source played major role in these transformations. As the need for commodification increased pressure to lower the price, existing software business models couldn’t cope with the increasing competition from the free software. Model started to changing into *aaS. Not much longer after that the open source technologies stopped being perceived as a disruption but rather as a competitive advantage.

Now it’s a good time to go back to Roman’s post. Where I agree with him is the fact the utility computing provides excellent environment for OSS to flourish. And increasingly complex world needs to be build on stable and open APIs, protocols and standards. Where my view differs is possible solution to the ‘problem’ open source is facing.

First I don’t really see a problem in the bazaar style development in the cloud perspective. Cloud computing, as it stands right now, is still in its infancy. And although we started calling our servers instances, talking about OS-level abstraction/virtualization and focusing on the streamlining provisioning, we are pretty much repeating what we’ve done before. Just changed the business model and created new horizons. Despite my reluctance, from perspective of innovation the future of the cloud computing is better characterized by platforms (like Google App Engine) – not IaaS as we perceive it today. For me underlying technology does not longer matter. I’m happy to leave so-called Cloud OS to existing providers. Because if I use the analogy of the internet connectivity and hosting, they are going to be cannibalized by their own approach. Bought, resoled and aggregated.

Second, I would like to use Roman’s terminology and change the outcome from two colliding solutions into their natural progression.

  • For me Benevolent dictators are just a logical mark of the evolution itself. Leading providers will keep their stack closed for a certain (very limited) time, no matter if it’s build on open or closed source technologies. But with the growing demand to facilitate the interoperability and facing increasing competition at the same time it’s predictable they will open-source their complete stacks in order to utilize their momentum. In order to stay on top of the race, existing players will be ultimately forced to initiate commodification of the cloud services itself and expand their offering. For this to happen there are still some prerequisites we’re missing – at least standardisation of APIs/models behind IaaS and unified measurement of the cloud mass/computing power/matter. The signs of this development are already visible.
  • The new model will eventually follow. The rational behind it is very simple. Industry wouldn’t be any longer able to build solutions on top of the increasingly complex architectures and face lack of the skilled professionals like you and me. As Roman pointed out, change itself will bring rise of conflicting opinions. In past, we managed to get over the virtual memory allocation, running our code within the virtual machines and cloud computing itself (as it’s just happening). But all these concepts addressed only existing problems. Now we’re facing new challenges. Building stones are already here – take *aaS, SOA, REST, hypermedia, microformats, orchestration and who knows what else, put it together and whoever will get the simple result will deliver it as a winner. Open source will definitely play important role not just as a facilitator for competitive advantage, but also as a instrument for companies to off-load their products when they become commoditized.
To summarize my rather long post, I believe the question is not how cloud computing will affect the movement. In my mind there only one alternative: is the open source going to have same affect to production of resources and services as a mass production had on commodification of physical goods? Cloud computing is certainly showing us a way. But to find the answer we must adopt other aspects of our society.
Sorry, but have to leave this one open… There are much better qualified people to talk about it.

To summarize my rather long post, I believe the question is not how cloud computing will affect the movement itself. In my mind, open source complements cloud computing, exactly as mass production allowed commodification of physical goods. The question is whatever the cloud is going to provide future platform for open source concept adoption even outside the IT industry? Social networking is certainly showing us the way. But to find the answer is outside the scope of this post…

[1] This analogy perfectly reflects my view on the private clouds.

How volatile Compute Clouds are supposed to be?

Yesterday afternoon, while relaxing in the garden I’ve suddenly realized why – in my mind – there was always a bit of hesitation to brand some of the emerging offerings as a true Cloud Computing. Amazon EC2 has always ticked the box. Why not the other providers? I knew the answer is going to be simple, but it took me quite a long time to figure it out.

Watching the sky where the strong winds were playing with the masses of condensed water vapor I’ve suddenly realized it’s not just a name these two phenomenas share together. It’s also volatility. Faithfully to their atmospheric cousins, compute clouds are definitely not representing static set of resources controlled by its users. Due to its nature where ‘perishable and intangible’ computing power is shared among multiple tenants to improve utilization rates, and therefore significantly reduce the cost, compute clouds are not just elastic but also indistinct, measured only by the mass of the resources currently allocated.

Having said that, I am obviously not suggesting Amazon EC2 resources are less resilient. Quite opposite. It’s simple a matter of the architecture where systems are designed for failure which at the end of the day will result in less failures affecting day-to-day business operations. Our personal experience might prove it, but nevertheless it’s always win-win situation if such a pattern is contained within the service itself. In this perspective I can see clear difference between Amazon EC2 and the other offerings. So until we’ll get more of the cloud nature, and less of re-branded VPS provisioning system, it’s going to be tough call.

Let’s try not to scale-in the IaaS resources or make them ever-so-resilient, approach more applicable within traditional infrastructure provisioning, but rather design our systems around this characteristic. And if you can’t get on with the indistinctive nature, try to have a look into a growing range of Platform as a service solutions. It’s a thin dividing line, I know, but the important one.

Cloud computing security (it’s not something new)

Following the rocket like boom of the cloud computing by the end of 2007, countless questions have been asked about security aspect of such a solutions. For many businesses this concern may overshadow other benefits – like agility, cost effectivity or scalability. This post is my reflection on true considerations one should take into account when moving into the cloud; all in perspective of the small to medium size businesses.

Many articles and studies casted a dark shadows on the general idea of on-demand provisioning of infrastructure. And they are right in one perspective: if you are not able to provide adequate security measures to your local hosted data or solution, you won’t be any better in the cloud (well, almost). Added remote access will only exponentially increase number of the potential intruders. But where this shadows do not reveal complete truth is the fact the lack of security is very often given by inability or negligence in businesses itself to establish adequate security. Which is not only the problem of small business, as we learned last year (2008) by series of blunders going all the way up to the British government.

Going back to the security of the cloud offering, where increased number of the security threats is anticipated, providers are (hopefully) taking preventive measures in place which we, regular users, wouldn’t be able to afford locally, especially in situations where the expectation is to bear the upfront cost of such a protection – no matter if it’s a physical equipment or operations staff (it’s up to you to pick the one more expensive for your business). As there’s no vendor able to address all the possible aspects and requirements, many of them choose openness to allow partners to provider added services. Perfect example of such a cooperation is the community surrounding Amazon AWS. Service aggregators will and have already started filling in the missing picture.

Other reactions are continually disputing physical security of the cloud computing and how such an anonymous solution can replace traditional collocation, dedicated or managed hosting services. It may sound bold, but I feel confident to say not only it can replace them but it certainly will, unless their are proactive in their offering. Based on personal experiences with even the most reputable companies on the market today we have to accept the accidents do happen, always due to the good old human factor. Especially when operation support is focused on an individual and very often not related resources, rather than anonymous blobs managed as a whole. Luckily traditional hosting market is not sleeping and we can already see different cloud based services coming from companies like Rackspace (Mosso) on one side and UK2 (vps.net) on another. To polarize the opinions a bit more I can’t wait who will knock in a final nail in the coffin of the companies refusing to change by introducing hosting platform provisioning on top of the existing clouds.

Due to the varied nature of the different cloud computing services it would be outside the scope of this post to list all the different security concerns, recovery scenarios and long-term viability options. This make selection of the provider important task, but the point is the process itself hasn’t changed so much compared to what we already know. Cloud computing is changing IT as never before, but it’s not technical rules that are changing (they’re evolving), but the business model is where the revision is being done; the rest is just a reflection of it.

RESTEasy integration with JBoss Microcontainer

With the JBoss JAX-RS implementation RESTEasy reaching version 1.0.1.GA I have finally managed to switch over from Jersey. Because my latest project has got all internal services and framework components represented as a JBossMC beans I needed a simple way how to export them directly as Restful resources without requiring any unnecessary boiler plate code.

I got inspired by Spring integration (which comes as part of the RESTEasy distribution) and wrote own resteasy-int-jbossmc which can be used under JBoss AS 5.0. Main difference is the application context scope – with Spring it has to be deployed as part of the same web application, whereas when deployed under JBossAS beans are instantiated within the microcontainer of the whole application server. To prevent scanning of unnecessary code I had to use feature called scoped kernel/controller and force restful resources to be scoped into own context, where they can be easily located.

Configuration (done in web.xml) then looks like this:

<web-app>
    <display-name>Archetype Created Web Application</display-name>

    <!-- configuration -->
    <context-param>
        <param-name>jbossmc-int.name</param-name>
        <param-value>APPLICATION</param-value>
    </context-param>

    <context-param>
        <param-name>jbossmc-int.qualifier</param-name>
        <param-value>demo-app</param-value>
    </context-param>

    <!-- RESTEasy bootstrap -->

    <listener>
        <listener-class>org.jboss.resteasy.plugins.server.servlet.ResteasyBootstrap</listener-class>
    </listener>

    <!--
       !! Integration has to be configured after Reasteasy Bootstrap !!
    -->
    <listener>
        <listener-class>net.laststation.tools.resteasy.plugins.MicrocontainerContextListener</listener-class>
    </listener>

    <servlet>
        <servlet-name>Resteasy</servlet-name>
        <servlet-class>org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher</servlet-class>
    </servlet>

    <!-- seems like mapping for now doesn't anything else than /* -->
    <servlet-mapping>
        <servlet-name>Resteasy</servlet-name>
        <url-pattern>/*</url-pattern>
    </servlet-mapping>
</web-app>

The integration is configured by jbossmc-int.name, specifying scoped context you want to use (APPLICATION is recommended), and the jbossmc-int.qualifier with he scope name itself (demo-app in this case). Also, please, note that ResteasyBootstrap must be configured first as it exports object required by MicrocontainerContextListener.

To export objects is then as easy as deploying them to microcontainer. Can be done using BeanMetadataBuilder, annotation or XML deployment descriptor as demostrated here:

<deployment xmlns="urn:jboss:bean-deployer:2.0">
    <bean name="library" class="net.laststation.demo.model.Library">
        <annotation>@org.jboss.metadata.plugins.scope.ApplicationScope("demo-app")</annotation>
    </bean>
</deployment>

The bean is exposed to MicrocontainerContextListener by annotation @ApplicationScope, which in this case specifies name ‘demo-app’.

Known Limitations

JBoss Microcontainer kernel scoping support at the prototype stages. Also similar to the Spring integration, Resteasy Servlet Dispatched must be mapped to the /*, otherwise it won’t work.

Download

Source code is available within my public BitBucket respository.

JBossMC – Scoped Kernel

Building the services using JBoss Microcontainer isn’t complicated at all; create necessary classes and wire them together using XML descriptor or couple of annotations. But unless you’re building just a very simple components, you might want to be able to hide the implementation and expose only the special interface (public) beans to the outside world. JBossMC has handy feature to support this using scoped kernel / controller.

Let’s start with the example:

<deployment xmlns="urn:jboss:bean-deployer:2.0">
	<bean name="sample1" class="net.laststation.demo.mc.SampleBean">
    	<annotation>@org.jboss.metadata.plugins.scope.ApplicationScope("testApp")</annotation>
	</bean>

     <bean name="myService" class="net.laststation.demo.mc.MyServiceDemo">
        <property name="holder"><search bean="sample1" type="leaves"/></property>
     </bean>
</deployment>

In this deployment we’ve got a sample bean annotated by @ApplicationScope which will force sample1 to be instantiated within the application scope testApp. Publicly exposed service, represented by myService, requires this object, but instead of the regular injection it must apply the search method – using the XML element (as shown above) or annotation @Search, in both cases specifying the bean identification and required strategy.

Search strategies are implemented within the package org.jboss.dependency.plugins.graph and in general consists form the two kind of implementations, based on only the local scope or hierarchy. Current list includes:

  • DEFAULT
  • LOCAL
  • TOP
  • PARENT_ONLY
  • PARENT
  • DEPTH
  • LEAVES
  • WIDTH
  • CHILD_ONLY_DEPTH
  • CHILD_ONLY_LEAVES
  • CHILD_ONLY_WIDTH

If you want to break the example, just change the search type to one that doesn’t take children scopes into account – for example TOP.

Below the line

Please note, the scoped kernel implementation is currently at the prototype stages. I wouldn’t expect significant changes in the functionality but configuration might be ‘adjusted’.

Personally, I would love to see support for the scope definition other than using annotations. Such an improvement would be much more coherent in the perspective of the meta data builder and XML configuration, where for example might be a place for the XML element <scope>.

Second comment goes to the requirement to use the search method instead of injection, for which the specification of the search strategy might be equally adequate.

JBoss AS 5.0 and Beyond

UPDATE 28-05-2009Release of JBoss AS 5.1 has made this blog post obsolete and because I don’t longer feel any urgency nor think that is appropriate to comment on RedHat release or quality control process I’ve decided to retract the text.

Mercurial hosting and bitbucket.org

Having spent some time this week resolving problems with Subversion repository due to poor network performance I’d like to intensify my efforts towards Mercurial (Hg). In this blog post not just the application itself, but rather introduce hosting capabilities and one of the projects from it’s biosphere.

No matter how decentralized version system might be, hosting is very important part of the infrastructure. Easiest way how to share Hg repository is to publish it using HTTP protocol. This can be easily achieved through CGI script called hgweb and in detail described in the document Publishing Repositories. To give you some idea how the interface look like, the list of some well known public sites follows.

Success of a project is very often given by the quality of available solutions. Perfect example in Mercurial world is Bitbucket.org, the code hosting site with social aspects, which allows you to host up to 150 MB for free, with unlimited number of the public and maximum of one private repositories. These limits can be increased by choosing one of the available commercial plans, starting with 5 USD a month for 500 MB quota. The public repository concept is key to understand social aspects of this site, which gives you possibility to track changes in different projects and actively participate. I wouldn’t mention this project if it would be just for code hosting. Integral part of bitbucket is full featured wiki and bug tracking system, very similar to my personal favourite application – Trac. Everything packaged in snappy web interface, easy to use and exposing even advanced Mercurial functionality – like change queues.

I definitely recommended to try bitbucket.org, even if you just want to get more comfortable with Mercurial. As a new service there’s still a lot of functionality to be added, but that doesn’t detract from its qualities.

My only wish in regards to the Bitbucket might be provisioning of some background information about the company behind, future plans, and (fingers crossed) possibility to make it open source! That would ultimately establish this project as one of the fundamental companion products for every developer.

Links:

Generic JSF Converter for Seam

JBoss Seam provides excellent support for mapping of the managed entities back and forth between the select items and the actual entities. Tag s:convertEntity supplies JSF converter which renders option values and consequently translate them back, using Entity Loader (either Hibernate Session or EntityManager), to the appropriate entity. Whole process relies on the provision of the entity identifier. This is indeed very powerful solution, but unfortunately so far I’ve haven’t got much chances to use it, except the training or with very simple applications. In most scenarios, the problem comes with strict separation of the presentation layer from business logic and therefore persistence support. And without Entity Loader, there’s not much fun with s:convertEntity.

Solution to this problem is quite simple – introduction of the custom JSF converter, where only requirement is to implement javax.faces.convert.Converter interface. There’s plenty of examples available on-line or you can refer to Chapter 15.5 of JSF In Action. If it’s for a single use, this is perfect solution. A bit more systematic approach to keep the code manageable is to create a generic converter.

This particular solution is based on the assumption there’s a collection of objects of the same class, and the need for identifier distinguishing these instances. Following the same approach as with s:convertEntity the first step would be to introduce thecustom Facelet tag.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE facelet-taglib PUBLIC
  "-//Sun Microsystems, Inc.//DTD Facelet Taglib 1.0//EN"
  "http://java.sun.com/dtd/facelet-taglib_1_0.dtd">
<facelet-taglib xmlns="http://java.sun.com/JSF/Facelet">
  <namespace>http://laststation.net/utils/jsf</namespace>

  <tag>
    <tag-name>convertGeneric</tag-name>
    <converter>
	  <converter-id>net.laststation.utils.jsf.converter.GenericConverter</converter-id>
    </converter>
  </tag>
</facelet-taglib>	

Save this file as META-INF/cg.taglib.xml and using namespace you can refer it within your XHTML view. Next step is to implement GenericConverter. Example follows.

package net.laststation.utils.jsf.converter;

import static org.jboss.seam.ScopeType.STATELESS;
import static org.jboss.seam.annotations.Install.FRAMEWORK;

import org.jboss.seam.annotations.Name;
import org.jboss.seam.annotations.Scope;
import org.jboss.seam.annotations.Install;
import org.jboss.seam.annotations.intercept.BypassInterceptors;
import org.jboss.seam.annotations.faces.Converter;
import org.apache.commons.logging.LogFactory;
import org.apache.commons.logging.Log;

import java.io.Serializable;
import java.util.Collection;
import java.beans.IntrospectionException;
import java.beans.PropertyDescriptor;
import java.lang.reflect.InvocationTargetException;
import javax.faces.component.UIComponent;
import javax.faces.context.FacesContext;

/**
 * Supports conversion of an object to/from an object
 *
 * @author radimm
 */
@Name("net.laststation.utils.jsf.converter.GenericConverter")
@Scope(STATELESS)
@Install(precedence = FRAMEWORK)
@Converter
@BypassInterceptors
public class GenericConverter implements javax.faces.convert.Converter {
    private Log log = LogFactory.getLog(GenericConverter.class);
    private String identifier;
    private Collection collection;

    public Object getAsObject(FacesContext ctx, UIComponent component, String s) {
        if (s == null)
            return null;

        if (collection != null) {
            for (Object item : collection) {
                String id = getItemIdentifier(item, s);

                if (id != null && id.equals(s)) {
                    return item;
                }
            }
        }

        return null;
    }

    public String getAsString(FacesContext ctx, UIComponent component, Object o) {
        return getItemIdentifier(o, identifier);
    }

    protected String getItemIdentifier(Object o, String property) {
        PropertyDescriptor desc;
        Object result;

        try {
            desc = new PropertyDescriptor(property, o.getClass());
            result = desc.getReadMethod().invoke(o);

            return result.toString();
        } catch (Throwable e) {
            log.error("Unable to get object identifier!", e);
        }

        return null;
    }

    public Collection getCollection() {
        return collection;
    }

    public void setCollection(Collection collection) {
        this.collection = collection;
    }

    public String getIdentifier() {
        return identifier;
    }

    public void setIdentifier(String identifier) {
        this.identifier = identifier;
    }
}

Implementation is quite straightforward, following the same principles as I’ve shown in the post Extending Seam components. It’s important to keep this component stateless (or at least bound them to EVENT scope) to make sure the supplied collection and the identifier are used only within a single call. Provided code is nothing close to perfect, it’s supposed to serve as an example to various different implementations. Also it’s supposed to be an example how easy is to write custom Facelet tag.

To use this converter just place code similar to following example where appropriate:

		<cg:convertGeneric collection="#{mycomponent.countries}" identifier="code"/>
	

Where countries is collection like List<Country> countries, and code is property of the objects within it. Using all the power of EL expressions and Seam functionality – like factories – there’s a plenty of use cases for this converter.

Complete source code is available for download as maven project (see links below). This example omits use of JSPs, because I hope nobody is really using them these days – especially with Seam projects.

Links: