Friday, December 17, 2010

Authentication and Authorization the CXF Way

I've been working recently on a number of tests and a demo showing how CXF endpoints can be protected by having consumers relying either on Basic Authentication or WS-Security UsernameTokens authenticated and the authorization rules enforced.

Often enough, when confronted with the problem of making sure the authentication and the authorization works, many users think of using either the J2EE-style authentication (configured via web.xml) or Spring Security.

The former option does not really work with WS-Security and sometimes may not be easy to configure in a fine-grained fashion, besides one may want to defer the role verification until just before the actual invocation occurs. The latter option can let you do it all, given how advanced the Spring Security is. If you're a Spring Security savvy person then wiring it in to secure the endpoints against some arbitrary WS-Security tokens can be a child's play :-) but one have to admit that it might be a bit 'intimidating' sometimes and in some cases Spring is simply not used in a given project. Additionally, making it work in OSGI may present quite a challenge on its own.

The other thing I've heard of is that users dealing with WS-Security may not necessarily want to deal directly with the WSS4J library at the level of their CXF interceptors (by the way, please subscribe to the Colm's blog for more information about WSS4J and other security-related thoughts and updates).

So in the end a number of CXF security interceptors has been introduced to make it as easy as possible for CXF users to secure the endpoints at the authentication and authorization levels.

Additionally, a CXF-specific UsernameToken has been introduced which at the moment encapsulates the information obtained from the WSS4J UsernameToken.
As you can see, it implements the SecurityToken interface with the idea being that more token types will be supported in time. For example, an AbstractSecurityContextInInterceptor converts a token to a CXF SecurityContext instance by delegating to subclasses which know about a current token type, with
AbstractUsernameTokenInterceptor being one such subclass. In the end, say a concrete AbstractUsernameTokenInterceptor implementation only sees CXF UsernameToken which it needs to use somehow for authenticating and populating a current Subject.

The Subject instance will be wrapped in a CXF SecurityContext which will be used later on to do the authorization or further wrapped by JAX-RS or JAX-RS SecurityContext objects and injected into service beans. You may want to use SimplePrincipal and SimpleGroup helpers if needed when creating custom Subjects.

Note that CXF ships two SecurityContext implementations which attempt to figure out which Subject Principals are roles. DefaultSecurityContext assumes Group instances represent primitive or group roles, while RolePrefixSecurityContextImpl find the Principals by checking if their names start from the configurable role prefix, example, from "ROLE_" (this post has some more information).

So far so good - the authentication has been taken care of if you use WS-Security. If you don't them the authentication will occur earlier on at the container entry level, with or without SpringSecurity involved. However, after working on the container-level authentication demo last week it became apparent that CXF needs a utility JAASLoginInterceptor. You only need to tell it the name of the LoginContext to resolve and optionally provide a role prefix name in case Subject does not use Groups to represent the roles. You may also need to override its getCallbackHandler method returning NamePasswordCallbackHandler instances by default, for example, this custom interceptor ensures it can handle Jetty ObjectCallbacks for setting the password.

Another good news is that the JAASLoginInterceptor will work equally well with CXF UsernameTokens (created during WS-Security-related requests) and AuthorizationPolicy capturing Basic or Digest authentication details.

Next we may need to enforce the authorization rules. SimpleAuthorizingInterceptor can be injected with a method names to roles map, while SecureAnnotationsInterceptor takes it one step further and lets users just inject a reference to a secured object and it will figure out how individual methods are supposed to be secured; it checks RolesAllowed annotations by default but one can provide the annotation class name used to specify the roles.
If you have roles stored in say the database then you may want to extend AbstractAuthorizingInterceptor and override its getExpectedRoles method.

If you are securing JAXWS or JAXRS endpoints, you only need to add one or two interceptors, say an optional JAASLoginInterceptor and SimpleAuthorizingInterceptor. When securing JAX-RS endpoints with these interceptors you may also want to add a custom CXF out fault interceptor, for example, like this one, if you want 403 or 401, instead of 500 being returned; alternatively you can let the exception propagate up to the servlet level when possible and deal with the ServletExceptions at the filter levels.

The JAX-RS frontend also ships two utility filters wrapping JAASLoginInterceptor and both authorization interceptors, The one which wraps the JAASLoginInterceptor returns 401 by default and adds a realm name to the WWW-Authenticate response header if needed. It also lets users specify the resource URI where the client needs to be redirected to instead of getting 401. If needed a simple subclass can ensure unrecognized callbacks are dealt with the same way it's done with JAASLoginInterceptor. Here is a sample filter .

Finally, some more information on how to configure the interceptors. Have a look at this beans.xml. The test assumes the authentication is managed by the container, for example, one can configure a Maven Jetty plugin to enforce it. The web.xml requests the authentication only and see how SimpleAuthorizingInterceptor and SecureAnnotationsInterceptor are set up to do the rest - the authorization.

This beans.xml introduces the JAASLoginInterceptor (and the JAX-RS filter) into the picture. I think it would be fare to say it is simple to setup both the JAAS authenticator and the authorization filters in CXF. And guess what - this very configuration can make your application secure without changing a bit in the standalone server, the servlet container and the OSGI container such as Karaf.

The varying part is ensuring the server/container sees the JAAS configuration such as this one (FYI, "BookLogin" is the name injected into JAASLoginInterceptor, while BookLoginModule is wrapping a Jetty PropertyFileLoginModule).

In my test I'm passing a system property "java.auth.security.login.config" to the test server process. In Karaf one needs to deploy a bundle containing the JAAS config. The bottom line is that the issue of bootstrapping the server with this information is an issue of its own.

So what do you do in the end once you set these new CXF interceptors ? You can just relax and see your secure CXF application working like a charm irrespectively of where it is deployed to :-)

Enjoy !



Link

Talend Service Factory Examples

As Dan has already mentioned, we have created a number of documented examples which have been tuned to run within the OSGI Karaf container shipped as part of the Talend Service Factory release.

The OSGI container in the release is really an optimized OSGI-aware Web Services stack with only the most relevant bundles being loaded. We've also tried to take care of minor issues such as ensuring that the default HTTP port is 8080 and that the HttpService context is set to "/services" by default as opposed to "/cxf". You can set these initial settings in 30 seconds yourself - but don't you want the demos just to work without having to know how to configure pax web and cxf osgi properties from the very start :-) ?

We have some serious plans to make sure CXF plays really well in OSGI - please keep an eye on the CXF dev list. We will want to make sure the CXF services and indeed the consumers just work in OSGI.

Additionally, most of the examples provide options for starting the services from the command line and within the servlet container.

Next I'd like to actually spend a bit of time and describe what examples we ship. What we have now is some classical CXF examples, such as the one demonstrating all sorts of CXF interceptors in action and additionally prepared to run in the OSGI container. The OSGI-fied example demonstrating SOAP over JMS specification is also there.

Finally we have started closing the gap which has existed in the area of JAX-RS demos and we've added 6 new demos.

Two first demos are called jaxrs-intro and jaxrs-advanced with the idea to show the basic JAX-RS features and then progress to a more involved example, not only showing more sophisticated JAX-RS features but also applying them to solving a reasonably interesting problem, traversing the Person family tree in a number of ways and updating individual Persons. These demos will be regularly updated.

The next 2 demos, jaxrs-jaxws-java-first and jaxrs-jaxws-description-first show how SOAP and RESTful services can be combined.
The first one shows how both JAX-WS and JAX-RS annotations can be combined on a single interface, how endpoints can share the same service bean and how both JAX-WS and CXF JAX-RS proxies can effectively use the same code for invoking on the services. The second one shows how SOAP services developed using a WSDL-first approach can be selectively exposed as RESTful services by applying an external CXF-JAXRS user model giving that modifying the interface generated from WSDL is not an option.

What we actually want to show with these 2 demos is that making this experience as seamless as possible is a top priority for us - we do want users to really like developing both SOAP and RESTful services with CXF.

The next demo, jaxrs-jms-http, shows how a typical JAX-RS HTTP server can be enhanced to receive the JMS messages with just adding few extra lines to the configuration. As noted here, you may want to do it if you'd like to preserve your investment in the JAX-RS but also make sure the service is capable of getting the messages from a variety of (transport) channels. This demo also shows how HTTP requests can be processed oneway with the server picking up the messages and forwarding them further to the JMS destination.

The last example is jaxrs-jaxws-authorization. This example shows CXF authentication and authorization interceptors in action, authenticating the users via JAAS and populating the CXF SecurityContext with Principal and its roles, and then enforcing the authorization rules for both JAX-WS and JAX-RS endpoints. The JAAS interceptor is only used when the demo is run within Karaf so that the Karaf JAAS context can be resolved. When the demo is deployed into the servlet container, the container-managed authentication is utilized to populate the current Principal.

This demo will be significantly simplified in the next release due to the recent related CXF enhancements (the topic of the next post).

You'll see many more examples added in time.

So please download the examples and give them a try. Note, you have to register in order to download the examples, this is due to the fact the examples content in general is considered to be a premium content at Talend, hopefully it won't be a problem for those who are interested.

Change the world and go home

I recall I was reading the Change the world or go home post from Dare Obasanjo and being a bit confused not about the content (which was interesting and thought provoking as usual) but about the 'or' bit in the subject.

I was asking myself, would it make sense to say "Change the world *and* go home" :-) ? I believe all of us working with the Open Source are changing the world every day, then we go home in the end of the day, and then we are back at it the next day, changing the world with our individual contributions, negligible in the scope of the big picture.

Of course, "or" had the meaning in the Dare's post. But I kind of like 'and' more :-)

Thursday, November 18, 2010

Sopera opens the development center in Ireland

Last week myself and my colleague Colm stepped in into an office that Sopera secured for us in Dublin, Ballsbridge. We can nearly see the IONA Building we used to go to not that long ago from our place. The view from its window is really nice.

We are delighted. Being able to get to the office and work with and talk to your colleagues but also work from home on a given day is the ideal situation for software engineers and it really does feel it is the beginning of the new journey for us, with the only way ahead from here - onwards, especially now that Sopera has been acquired by Talend.

Monday, November 15, 2010

CXF Auto Redirect Feature in action

I was working today on testing some CXF JAX-RS client code against Tomcat and Jetty servers. All was going well with Tomcat : after deploying a sample.war to Tomcat and doing :


WebClient wc = WebClient.create("http://localhost:8080/sample";
wc.getCollection(Book.class);
// print the collection


a collection of Books was printed in the console.

However this code stopped working once I deployed a sample.war to Jetty, using a Maven Jetty plugin. I spent some good few hours debugging it before I spotted that Jetty returns HTTP 302 to a "GET /sample" request with the Location header pointing to "http://localhost:8080/sample/", note the trailing slash.

So changing the code to


WebClient wc = WebClient.create("http://localhost:8080/sample/";
wc.getCollection(Book.class);


made it work for both Tomcat and Jetty deployments. One other option to deal with the redirects is to use one of the WebClient methods returning JAX-RS Response rather than a typed one, and check the status and then follow the new URI if needed.

But I found the following code working nicely in the end :


WebClient wc = WebClient.create("http://localhost:8080/sample";
WebClient.getConfig(wc).getHttpConduit().getClient()
.setAutoRedirect(true);
wc.getCollection(Book.class);

Monday, November 8, 2010

Being at ApacheCon 2010

I got a chance to attend to the Apache Conference in Atlanta last week thanks to Sopera organizing this trip. It was the first time I was at the ApacheCon, and it was an interesting and unique experience.

First, Sopera had a workshop during the first 2 days of the week, before the conference started, so it was a good opportunity for many of us in the OS team to meet each other and we had some interesting discussions along the way.

ApacheCon brings together an interesting mix of people, some of them having the commercial interest, some representing large commercial and government organizations adopting OS, and some working on OS projects because this is what they really like doing on their own time.

It was interesting to listen to some technical talks. And meeting new people and seeing the former colleagues I used to work with (with some of them I will be working again from now on :-)) was great. And of course, meeting CXF legends such as Glen and Benson was another highlight :-).

Wednesday, October 27, 2010

Give us your headaches

Have I already mentioned that I liked reading the Richard Branson's auto-biography ?

I honestly think it was a gem of a book which I picked up in the Gatwick airport on the way to Dublin from Minsk. That is a story about the life journey told with a lot of humor, filled with some absolutely hilarious accounts of various events and with some very serious thoughts about doing business and contributing to saving the humanity.

You might want to ask, what is it all to do with the web services or software development given this post is not marked with [OT] ?

During his student days, Richard set up the Student Advisory Centre which consulted students on all the issues they could have, free of charge. One of their slogans was "Give us your headaches". This centre still operates today.

I thought, when I was reading about it, that to some extent, this is what the OS business model was partly about. In OS we are working on projects such as CXF and we are determined to make it all work really well, without any headaches, in the most demanding environments.

The OS business model is partially about providing the insurance that customers will experience no headaches when running such OS-driven projects in their productions. In fact this model has been proven very successful, by such top companies such as RedHat for example.

Are you considering to start using CXF but a bit concerned what will happen further down the road with all the OS development going on into it ?

Give us your headaches :-)

CXF : Here we come !

I'm happy to confirm that after a short break I'm going to have a chance to resume working on CXF and its JAXRS project in particular.

I'll join Dan and the rest of the Sopera OS team and we will be determined as ever to bring you the best production-quality web services framework around.

And with your help we will make the CXF users and dev lists the "hippiest place to be", the phrase is borrowed from a Richard Branson's auto-biography.

CXF won't be the only effort we will focus in Sopera but it is going to be one of the main ones.

Stay tuned.

Goodbye JBoss

So after less than 6 months after starting to work for JBoss I decided to quit and accept an offer from Sopera. In hindsight, I'm thinking that talking about the career publicly is not always the very best idea - spending such a short period of time at one of the leading company in the industry is not something that will improve my CV.

Back in April/May I was indeed looking forward to that great opportunity and it was not a stop-gap measure by any means. But CXF and CXF JAXRS are calling me back - it is next to impossible to resist. I'll blog about it next but this post is about JBoss and I'd like to talk about it a bit.

Just 6 months ago I had a fairly vague idea about what JBoss were doing, apart from knowing that they were part of RedHat, that it was an application server and that their JBossWS project was providing a CXF JAXWS integration option. And of course I knew about RestEasy.

In the reality, JBoss is a very live, active and progressive project, with a lot of very clever and innovative people working on it and who are really passionate about JBoss. And they have a very open and democratic environment with everyone being able to express their thoughts aloud.

I've promised in my previous post I'd link to various JBoss subprojects. Here are the links to some of them :

JBossWS : Web Service Framework for JBoss AS
RestEasy : JAX-RS implementation with various features
PicketLink : STS, etc
JBossTS : While many are saying that WS transactions support is next to impossible to provide this team just does it
HornetQ : Putting the buzz in messaging


As far as I'm concerned it is obvious I've given away the real opportunity with a great company. That said, the life is about many opportunities, and I'm about to pursue the new and exciting one, with the young and innovative company.

Goodbye JBoss, Hello Sopera !

Monday, May 24, 2010

The New Beginning : JBoss

Today I've started working for JBoss, the division of RedHat.

This is a truly new beginning for me and I'm looking forward to my new career in JBoss and RedHat. I'll have a chance to work with and learn from a lot of great engineers and contribute to a number of interesting projects.

It is a big new world out there. RedHat Linux, JBoss, ws and restful services, clouds, virtualization, messaging, transactions, osgi, etc... I'm optimistic and hopeful it will be a great ride.

From now on I'll be linking to various RedHat/JBoss related projects, news, etc.

I'll continue working in a web services area. I'll be involved in a JBossWS project. I'll also have a chance to work with RestEasy and possibly contribute to it; it is a powerful JAXRS implementation which has been leading alongside with Jersey and it is also likely to become a home for realizing and implementing a number of pragmatic ideas coming from the Rest-* effort.

I also have a message for those of you who have decided to try or use CXF JAXRS. It is a project I've put a lot of effort into and I'm really keen to continue supporting CXF JAXRS users which is what I'll be doing though the time I'll be able to spend on it will be limited to after-work hours and weekends. I think CXF JAXRS will do well - it's become quite solid with respect to helping CXF users with developing REST-based services. The development effort which will go into it will become quite limited - but I hope it will live.

Stay tuned and have fun !

Wednesday, May 19, 2010

New Kid On the Block : JBoss OSGI

I've been reviewing today the JBoss OSGI documentation and I've been really impressed. They can work with existing containers such as Equinox and Felix but can do well on its own too. Check the documentation - it is very well written.

For some of you a 'new kid on the block' may not sound right if you've worked with or heard of JBossOsgi already :-).

It's likely to affect the existing balance on the market of OSGI-aware frameworks. Watch this space.

Tuesday, May 11, 2010

OT : bbc.co.uk and tut.by

Time and time again I'm being very impressed by the quality of [bbc.co.uk|http://www.bbc.co.uk/]. It is just possibly the best web site which I've seen. Their "Live Football" section is the best one too :-)

The [tut.by|http://www.tut.by/] (Belorussian news portal) is quite impressive too, not as powerful as the BBC one - but good nonetheless.

Friday, March 26, 2010

Converting FIQL expressions into SQL queries in CXF JAXRS

I've done some more work on the search extensions so that users can easily introspect a given SearchCondition instance representing either a primitive or complex query and also added a utility SearchCondition.toSQL(String tableName, String... columnNames) which can be used to convert this search condition into an equivalent SQL query to be subsequently executed against a database.

For example,


// find all conditions with names starting from 'ami'
// and levels greater than 10
SearchCondition sc =
fiqlParser.parse("name==ami*;level=gt=10");
assertEquals("SELECT * FROM table
WHERE
name LIKE 'ami%'
AND
level > '10'",
sq.toSQL("table"));

// find all conditions with names starting from 'foo'
// and those with names not ending with 'bar' or
// levels greater than 10
SearchCondition sc2 =
fiqlParser.parse(
"name==foo*;(name!=*bar,level=gt=10)");
assertEquals("SELECT * FROM table
WHERE
(name LIKE 'foo%')
AND
((level > '10'")
OR
(name NOT LIKE '%bar')),
sq.toSQL("table"));




Here are some clarifications. First, note that the above code shows that a FIQL parser creates a search condition but it is a unit test code, the actual user code will use a SearchContext instead in a type safe way, providing a bean class to it. SearchContext will create an instance and will inject into it the property values, those provided and extracted from the original FIQL expression.

This provides for a possibility to protect against possible SQL injection attacks at the Java and the bean validation levels. For example, if a Book bean has an integer 'id' property then there's no way an unsafe FIQL query such as 'id==DROP%20users' can result in a 'DROP users' value injected due to a NumberFormatException.

Additionally, bean setters may have some custom validation logic and the CXF FIQL parser may also get enhanced to check for the bean validation annotations.

Perhaps you might want to give it a try and see how safe this option of auto-converting FIQL expressions into SQL queries is ?

In the end of the day, the good thing is that you actually do not have to use
SearchCondition.toSQL to get the SQL query. Suppose you are still concerned about the security or see that this method is not going to help you with building the advanced SQL queries which you actually use in the production, may be you are using java.sql.PreparedStatements or perhaps you don't even use a relational database but rather work with say Cassandra. If it is the case then just introspect a SearchCondition by getting to all the subconditions it has, and build a query which suits best.

Saturday, March 20, 2010

XOP support in CXF JAXRS

An XML-binary Optimized Packaging (XOP) is a W3C specification which is mainly associated with a SOAP Message Transmission Optimization Mechanism (MTOM) but is equally applicable to plain XML.

XOP is actually a very good specification which provides for the inclusion of binary data in XML payloads via links to co-located or external parts and thus facilitates the processing of such payloads by technologies like XSLT or even XMLSecurity, this article gives a pretty good description.

A CXF user has recently tried to have a single Java service class serving both SOAP MTOM and 'basic' HTTP multipart-related requests. Now, it is possible to do it, but I found it was not quite appealing so to say for users having to create either MTOM or multipart/related (with no links from the root part) payloads for the whole idea to work out. In addition, users have to add CXF JAXRS Multipart annotations to every method parameters corresponding to individual multipart parts which was not ideal given that the same user was also experimenting with a document-first approach.

So it took me virtually 1 day to add a XOP support to CXF JAXRS, thanks, as usual, to the fact that CXF has it all, specifically, attachment interceptors already deal with MTOP/XOP and its JAXB databinding has all the related utility code required for serializing/deserializing XOP payloads. One would need to set a CXF Message.MTOM-ENABLED property on CXF JAXRS clients/endpoints, but I think it is ok even though a property name is SOAP related, but it is really just an indicator to CXF as to whether do XOP or not.

I think it is a good enhancement indeed :

- SOAP users doing multiparts (MTOM) can have a simpler option for reusing the same code base when trying to do HTTP/REST as well, the only restriction at the moment is that CXF JAXRS will not unwrap the complex data types, example, just have a single EmailPayload method parameter, as opposed to many parameters corresponding to EmailPayload parts, such as text, attachments, etc; it is better anyway IMHO

- it is just a nice to have feature for CXF JAXRS users to play with, as far the multiparts support is concerned

The only regret I have is that I started working on it late enough and thus did not have it merged into soon to be released 2.2.7.

Monday, March 15, 2010

WSDL - first, CXF JAXRS model - first !

So you're a SOAP developer who prefers starting the development of a new service from creating a WSDL document first. It is a WSDL-first approach that works for you, not a Java-first one. You are also interested in giving it a try and seeing what advantages following the RESTful style can give the users of your application, but would also like to keep a single code base.

If so then please read on this long post...

CXF makes it easy for developers to expose a single Java class as a SOAP and REST service, by sharing a service bean between multiple jaxws and jaxrs endpoints. The problem is, as far as you're concerned, that it requires one to follow a Java-first approach.

One option is for CXF to offer users an option to do a WADL-first development. CXF JAXRS is not quite there just yet but nothing prevents it now from providing a code gen command line utility and a maven plugin. This is an ongoing effort though and some UI tooling support would also be required. So, eventually, one would be able to use WSDL for developing SOAP services and WADL for developing RESTful services. This may or may not become a viable approach for users preferring a document-first style. Working on different documents (WADL and WSDL) might not make much sense, and creating a shared service class will not be possible.

The other advice which you, the SOAP developer, are most likely to hear today is as follows : you'd be better off migrating to REST/JAXRS altogether rather than sticking with SOAP. So the issue of figuring out how to do SOAP and REST the document-first style at the same time would be resolved. Even though there is some evidence that a number of individual and corporate developers may have done just that, I think this advice may not be the most effective option, as far as convincing those who have invested in SOAP and actually liking it no matter what that there could be a better option awaiting them. It is enhancing the runtime for it to be able to do the advanced stuff SOAP developers can do today with respect to the message-level security, multiparts and the document-first development and providing the easy path for SOAP developers to experiment with REST in the live production is what can convince the most conservative developers over time that the other approach (REST) may be adopted, partially at least.

Here is another option CXF could try: implement WSDL2, thus letting users define both SOAP and REST bindings in a single document which can be quite effective from the development point of view. Now, WSDL2 is said to be a better language than WSDL 1.1, but the question is : would an investment into implementing it be cost-effective ? Will it bring new users to CXF? And is it worth investing into it just for the sake of letting users do both SOAP and REST development the document-first way ? It does not seem it can do better that WADL in describing a RESTful application, given that it is more likely WADL can be enhanced even further while WSDL2 will not(not touching here on the interface vs resource centric approaches issue). Finally, as Dan Kulp says, a user demand is just not there, at least at the CXF users list. It can be quite a difficult thing to do, deciding on whether a given feature should be implemented depending on the existing or anticipated demand from users, sometimes expecting the users/customers always indicate to you what should be done next can stifle the innovation, but in case of WSDL2 it seems like the right thing to do.

This post has been long enough but I've tried to do some analysis there...So here is what can be done right now the real cost effective way (thanks to one of CXF users trying this option first) : generate your SOAP and JAXB classes from WSDL and then
'attach' some RESTful behavior to generated interfaces by relying on the annotation-free feature and finally have both jaxws and jaxrs endpoint sharing an implementation bean, with the jaxrs endpoint including the model describing the interface only. Example, imagine that no JAXRS annotations exist in this (generated) source interface. Next, just describe this interface similarly to the way it is done here. Finally add a jaxrs endpoint (see the one with a 'bookservice3' id) referring to the model describing this interface and the bean which implements it and you're done.

It would be mostly about making your code a bit more HTTP aware as opposed to magically turning it into a RESTful application, but it is an easy start and you may want to give it a try. And if it works for you then you might want to ask your friends from the UI team to have a tool generating CXF JAXRS models for you :-)

Saturday, March 13, 2010

CXF JAXRS Search Extensions

What is it that defines a user experience on the WEB ? Reading the information, submitting some information, following the links and searching.

How to let the users do the advanced search of the data your application makes available publicly ? One way is to let users type or transform their requirements into XQuery expressions, another option is to build a URI-friendly query language which will let users easily type it and link to the resulting URIs.

This post provides a good comparison between two approaches used by Microsoft Astoria and Google Base Data respectively. As you can see one can do some fairly sophisticated queries but they seem to be quite complicated for users be able to type them manually and in some cases are not easy on the eye. Both approaches provide for advanced search capabilities (to do with hierarchies, etc) but which may not be easy to implement.

Andy Michalec and myself have worked recently on implementing a search extension which will allow users to use a Feed Item Query Language (FIQL) to do the advanced search queries against CXF JAXRS endpoints. FIQL is a simple, very intuitive and easy on the eye URI-friendly language which lets users express fairly advanced queries.

Mark Nottingham writes simple and practical specifications. CXF now implements two specifications authored by Mark, RFC-5005 (partially) and the FIQL draft and I think CXF users will start getting some real value out of it.

Even though FIQL is a language originally meant to be used for searching Atom feeds, it is written such that non-Atom data can be searched too. So after Andy has spent an hour or so :-) on writing a quality FIQL parser, I just wired in a SearchContext into the CXF JAXRS runtime and now users can do the following :


WebClient wc =
WebClient.create("http://localhost:9080/books?"
+ "_s=name==CXF*;id=ge=123;id=lt=555");
List<Book> books = wc.getCollection(Book.class);


Here we create a FIQL expression allowing us to get all the books with names starting from "CXF" and ids greater or equal to 123 but less than 555. FIQL can let you do much more powerful queries but I think it is just a brilliant effort from Mark.

Note how simple the query is, here are the rules in a nutshell : it is only the first '=', the one which follows an _s query name (for the record, _search is also supported) which acts as a usual separator between a query name and its value, '==' means 'equals', while '=' in '=gt=', etc helps easily identify a condition such as 'greater than'. A user can quite easily type such an expression in a browser without some (web )UI gadget hiding the complexity of the language.

And here is the server side test code :



@Path("books")
public class Books {

private Map<Long, Book> books;
@Context
private SearchContext context;

@GET
public List<Book> getBook() {

SearchCondition<Book> sc =
searchContext.getCondition(Book.class);
List<Book> found = sc.findAll(books.values());
return found;
}
}


An injected SearchContext returns you a SearchCondition which can be a composite one and encapsulates a given FIQL expression, though nothing prevents us transparently supporting other languages in the future, such as XQuery, etc.

Now, at the moment the assumption is made that the data are kept in memory, but often enough the search requirements are passed down to the database layer. You can try to push SearchCondition as close as possible to the db layer but SearchCondition, having captured the FIQL expression can let you easily transform it into an SQL language expression if needed, ex, it can let you check all the conditions it encapsulates, and the values which will be used during the checks, etc.

I'm not sure yet if I'll merge it into 2.2.7-SNAPSHOT or not. May be I'll keep it in 2.3-SNAPSHOT and experiment a bit more and then merge it into 2.2.8-SNAPSHOT once the interfaces are deemed capable enough. Will also update the Atom pull server to let users search for very specific log entries, that what FIQL was created for :-)

Enjoy !

Wednesday, March 10, 2010

On the way to WADL-first development in CXF JAXRS

Yesterday I added a CXF JAXRS request filter capable of generating the Java code for consuming a given JAXRS resource on demand. I was about to finish it two weeks ago but had to postpone it a bit. Here's a test code fragment showing how to get a zip containing the source :


// Get the link to the source
WebClient wc =
WebClient.create("http://localhost:9080/petstore");
XMLSource source = wc.query("_code")
.query("_os", getOs())
.get(XMLSource.class);
String link = source.getValue(
"ns:html/ns:body/ns:ul/ns:a/@href",
Collections.singletonMap("ns",
"http://www.w3.org/1999/xhtml"));
// download a zip file
WebClient wcZip = WebClient.create(link);
InputStream is = wcZip.accept("application/zip")
.get(InputStream.class);
// unzip and compile it


Please see a testPetStoreSource test for more details. The _os query is optional, it is there to ensure you can get OS-specific line separators added to some of the generated classes (ex, the server is running on Windows, you'd like to run a test client on Unix). You can also add a _language query but it is only Java which is supported for now.

What is interesting is that a newly introduced CodeGeneratorProvider generates the source from a WADL instance which another provider, WADLGenerator, creates after introspecting a given JAXRS resource. The zipped source will contain the JAXB-generated classes from a WADL grammar section (if any) and the proxies (interfaces) representing WADL resources, including the subresources, with the resource methods linking to corresponding JAXB classes.

You can request that classes be generated from a WADL grammar section only by adding a _codeType=grammar query. Two other possible values here are 'proxy' and 'web', the former value is a default one at the moment with the latter one being not supported yet. I need to think a bit more about what sort of code can be generated for a _codeType=web. Suppose we have a resource named BookStore. What is the difference between a method called getBook (in case of proxies) and a method called get(in case of web clients) ? I'm also not sure yet what to do with cases where WADL resource contains multiple methods like GET but with different mediaType values, need a bit more time. In meantime if the word 'proxy' makes you nervous :-) then please just use _codeType=grammar to download the classes representing schema elements and proceed from there by using your JAXRS runtime's web client api.

How does CodeGeneratorProvider generate the proxies given that WADL does not provide for it ? Internally, it requests a WADLGenerator to add 'id' attributes to WADL resource elements representing root or subresource resources and to WADL resource/method elements. Example, wadl:resource/@id="{org.apache.cxf.test.RootResource}" and wadl:resource/wadl:method/@id="getBook".

This is really all what 'facilitates' the WADL to Java code generation which is exactly what CodeGeneratorProvider does. In fact, I haven't actually even thought originally about introducing such a provider but then I thought that a future command-line codegen tool allowing for a server-side code be generated from a WADL document will do exactly what CodeGeneratorProvider does, albeit it will generate the resource class skeletons as opposed to proxies. In fact, given that CodeGeneratorProvider is a regular CXF JAXRS provider, you can set a false 'generateInterfaces' property on it and you'll get a *server-side* code downloaded, perhaps you might want to use it to write your own added-value service interposing the original one.

CXF 2.2.7 will be out soon so please experiment with this new feature.

I'll need to think a bit more about the format for resource ids, perhaps it can be just "BookStore" but a future codegen tool can ask users to provide a package name. We can have JAXB classes acting as subresources that is why I've introduced an extended QName format initially. At the moment WADL does not allow internal method elements to contain ids (as opposed to global ones), hope this restriction can be lifted, alternatively it would be easy for a (UI) tool allowing users to create WADL documents to ensure resource methods are declared globally and then linked to from internal resources.

Seems like using resource and method ids for the purpose of allowing WADL-first development might make sense, it will preserve the simplicity of WADL the language.

Finally, why worry about the WADL-first development ? It is simple : IMHO it might help in making the RESTful approach toward developing web services being more attractive to many developers who prefer starting with building a picture of the future application first. So imagine your UI tool showing a nice view of how resources relate/link to each other and what formats and HTTP methods they support and schemas they're aware of, would not it be cool to be able to press a button next and have most of the code being generated for you ?

CXF is my favourite game

I was listening to the radio in the car this morning and I heard 'And I'm losing my favourite game' by The Cardigans. For some reasons, I'm often associating certain phrases from the songs I hear at a given moment of time with what I'm thinking or doing at the moment. So the idea of "CXF is my favourite game" sprang to mind and here I am, with all the time on my hands, blogging about it.

The last week was quite interesting. For the most of the last 20 years I've been at work virtually every day doing something, writing some code, discussing something, thus being presented with an opportunity (this is how I'd like to see our severance with the Fuse team) to rethink and take a break poses quite a challenge. But there's definitely something in it. For example, I've been to London many times but I felt the trip last week was the best one : you kind of feel the energy of the city and realize the world is so rich and there are definitely opportunities lurking out there.

I've been focusing mainly on CXF for the last couple of years. I don't know yet where my next career move will bring me to, but I'm feeling there could more coming from CXF, I'll try to keep contributing whenever a get a chance. The software world is big but I'd like to see CXF being a continuous success.

Friday, February 26, 2010

Higher and higher we are gonna take it...

I've been thinking today how to title this blog entry. First I typed "Sky is the limit" but spotted this page. Then I typed "The best is yet to come" but surely I found this one. So I ended up borrowing a phrase from the Killer's Confessions of the King and stopped being melodramatic.

It is the time to seek a new beginning and reflect a bit on the last 5.5 years I spent with IONA and Progress Fuse.

IONA was the company where I grew up as a software engineer. I was fortunate to work from the day one with the best engineers one can dream of working with, some of them being very well-known in the industry and some not but it is them from the latter group who brought me to that stage where seeing a failing system test passing was the most satisfactory thing one can imagine. It was the company where one could learn to compromise and move forward. And we had quite a few laughs all the way ! And VisIONAry t-shirts were the 'best' in the industry :-).

Working for IONA was the real thing. I enjoyed it. But I have to admit working in Progress Fuse on the CXF JAXRS implementation was probably the best part. It was the 'working on the line' kind of experience, the bleeding edge, it had it all ! Working on the open source projects gives one a chance to be a coder and an architect and lets the one to talk to users on the everyday basis. I'm not sure what can be better, as far as the software engineering is concerned.

Now it is time to think. Listen to U2. Start updating my CV. Visit London next week and hope Arsenal will not lose again (good we bought the city break package the previous week :-) as I'm feeling a bit frugal at the moment :-)) Continue supporting CXF users. And seek the new beginning.

Stay tuned and have fun !

Wednesday, February 17, 2010

Use your favorite Atom reader to view CXF logs

Andy Michalec and myself have worked recently on the Atom-based logging feature. Andy has implemented all the base classes and a push-style feature as well as wrote the initial draft describing how the push-style logging works. Myself has contributed recently by adding the initial code for supporting a pull-style logging.

Here is some more information about these features. They haven't made it into 2.2.6 nor will they make it into 2.2.7, etc but rather stay on the 2.3-SNAPSHOT trunk given that their code has been moved from the jaxrs frontend to a new rt-management-web component available on the trunk only.

I reckon the push style feature is already quite functional, you can try it by downloading a 2.3-SNAPSHOT and 'attach' AtomPushBeans to your jaxws or jaxrs endpoints by listing the loggers used by your code, as documented on the wiki. You may also find these links of interest :
beans.xml
and JAXRSAtomLoggingPushSpringTest, plus non-Spring JAXRSAtomLoggingPushTest and batch logging.properties. You can have log entries wrapped in a number of ways, as feeds or entries, with the content being added as an explicit entry content or an entry extension. Processing such entries is easy on the client side, with CXF you can do xmlSource.getNode('atom:entry/atom:content/*', LogRecord.class) or xmlSource.getNodes('atom:feed/atom:entry/atom:content/*', LogRecord.class).

Before actually moving to describing the pull-style feature, I'd like to mention that CXF can transparently ensure that Log4J or SL4J events can be captured back into JUL log records so you can get non-JUL records pushed/pulled as well.

Now, just today, I've finalized the first cut of the pull-style logging feature and I'm actually quite excited with the fact I was able to get a link to an endpoint-specific log feed from the CXF services page and see the list of log entries in Firefox, IE and a standalone Atom reader, by traversing to individual log entries (not in IE though which does not seem to recognize alternate links). Give it a try please, you actually might like it and it is very easy to try :-) !

Have a look at this beans.xml (see an atomPullServer bean which 'attaches' itself to a 'resourceServer' endpoint and relies on an 'atomServer' endpoint to expose the log entries) and the JAXRSAtomLoggingPullString test (see testFeed()). You will also need to add a dependency on the rt/management-web.

Some more details about the AtomPullServer. By default it is set to display 40 entries per page and keep 500 records in memory. It uses 'previous', 'next', 'last' and 'first' links to let users traverse across the whole range. The problem here is that I don't know of atom readers supporting these link types, let me know please if you do. Perhaps you do a simple atom browser yourself or easily deal with these links programmatically. May be we even add a basic browser in CXF over time.

You can inject into it a ReadWriteLogStorage implementation which can be used to offload the records to the external storage and persist them after restarts. Or just register a ReadOnlyLogStorage if you do not want your logging runtime to generate duplicate records if they're already being saved to an external file. At the moment, the latter storage requires a bit more effort as you'd need to write a file parser but it's doable and we may add few helpers later on. ReadOnlyLogStorage relies on the SearchCondition, stay tuned on the updates about using it in a regular JAXRS application code. Note that even if you already log to a file, you can still use the AtomPullServer to capture the most important log entries, there will be few of them in a healthy system so you can probably just set the max in memory size to 1000 and not worry about reading the same events from a file...

So this is what we have at the moment, give it a try please and help us to improve this feature.

Enjoy !

Tuesday, February 9, 2010

Support for JMS in CXF JAXRS

One of CXF JAXRS users have asked about the possibility of supporting JMS for a given JAXRS resource be able to get data over HTTP and JMS.

CXF has been designed from the ground up to support multiple transports so it was very easy to ensure CXF JAXRS could get the messages from JMS. Please see this beans.xml, it is just a matter of adding a transportId attribute to a given jaxrs:server endpoint, so if you have say a single service bean shared by both http and jms aware jaxrs endpoints you have multiple channels supported.

Now, you may ask, what is it all about ? JAXRS is about helping users to build HTTP-based RESTful services ? What about a "transport independence being a bug but not a feature" thing ?

I guess I could've said that REST is not about using HTTP, but it is just that HTTP, being the transport/application protocol of the WEB, is firmly associated with REST.

What really matters IMHO is that CXF JAXRS users will be able to see their 'investment' in JAXRS growing up. They've chosen to structure and annotate their resources in a way which makes it easy to expose them as RESTful services and being able to get data from different channels by relying on the underlying runtime is a good thing.

One of the reasons behind the 'longetivity' of SOAP is that users can get SOAP messages from JMS, in other words, they can easily tap into all those MQ stores with the help of the runtime.
Of course, one well-accepted alternative would be to use a routing engine such as Camel which would delegate from a jms channel to a given jaxrs endpoint. CXF gives you another alternative now.

What is common between CXF and Microsoft WCF

This morning I've read the interview with Don Box (thanks to this blog entry). I'm kind of unhappy with Twitter 'hijacking' a lot of people like Don expressing their thoughts aloud, I like reading the blogs, I can not get a lot out of Twitter one-sentences. I was 'following' Don since him contributing to his MSDN COM column, I wish himself and others could still find some time and blog...

So back to the interview. Here is what I liked most. Stefan asks :

" What is Microsoft doing from your perspective with regards to REST support? Do you see the two worlds coexist? Which role does which one play?"

Don replies and finishes with :

"One of the interesting things about Microsoft that took me a couple of years to understand was there is no place for religion in products. We build products for lots of different people and if someone wants to do REST, I want them to love our stuff. If someone wants to do SOAP and WSDL, I want them to love our stuff. Basically, it's our job to allow our customers to do what they want to go do with their software and the easier we can make that, the better. "

+1. This is what it is all about. This is what CXF is about. I love working on the CXF JAXRS project, JAXRS is a fantastic spec. I've also had some great experience in interacting with (IONA) customers (though mostly indirectly by fixing bugs :-)) who continue building advanced SOAP services even today. In CXF we are trying hard to ensure users can get the best of their chosen technologies/approaches, by combining them if needed.

I also liked how Don was talking about Mark Baker. In fact I was quite fortunate to have a long exchange with Mark where I was asking him a lot of newbie questions, it can even be found at www-archive@w3.org.