Jettison 1.3.1 has been released

Jettison 1.3.1 has just been released, please the Download page for more information about the latest fixes.

The two updates which would be of interest to CXF users (or those who rely on other stacks that ship the providers wrapping Jettison) are to do with making it possible to create formatted JSON outputs and making sure that the ignorable (empty) text content (reported by JAXB when processing the mixed content data) is blocked. The latter update, among other things, will make the WADL JSON representations look nicer given that they are produced from the wrapped XML payload.

Few other improvements made it into this release too, particularly those JIRAs which had the patches attached to them have been resolved.

I'd like to encourage the Jettison community to continue providing patches and as far as I'm concerned I can say that the patches will be eventually applied and the fixes will make it into the the future Jettison releases.

Happy New Year !

Associating user login names with OAuth Access tokens

The classical OAuth flow involves the authorized 3rd-party client magically accessing the end user's resources without even specifying a user name or id, for example:

GET /user/calendar

How can the server figure out which resource to return ? The answer is that the request will have an Authorization header with an access token key and in OAuth 1.0 the token needs to get the information about the end user authorizing a given 3rd party consumer associated with it at the authorization time.

With this association in place, the server will have an easy way to get to the name or id of the user used to authorize the client, and the way to do it in soon to be released CXF 2.5.1 is documented here.

I'd like to thank Glen for helping me with getting this enhancement done.

We are continuing working on the demo which will show most of the CXF OAuth support in practice. Please stay tuned to see how you can write secure OAuth applications with CXF :-)

Locator Support For RESTful Services in Talend ESB

Back in June I blogged about CXF JAX-RS providing a fail-over support for RESTful Java clients and mentioned some relevant work started in Talend.

Things are happening in the Talend land where all the interesting new features in Apache projects that our Coders team is contributing to are taken on board, analyzed and wired in the products real fast. And the newly released Talend ESB 5 brings a Locator support for RESTful endpoints and clients with the client-side Locator feature built on top of the CXF fail-over mechanism. Kudos to the ESB team for implementing it.

If you get a working HTTP-centric code then having such a code not hard-coding the endpoint address of the RESTful server which is part of the large WEB application and seeing it working even after a given server or the HTTP connection to it goes down will be encouraging.

Are you all into writing large RESTful applications end-to-end ? Have problems answering to your friends who like working with SOAP (they are still your friends though :-)) and tell you that with REST it's not possible to write Locator-aware fail-over capable clients ? If the answer to either of those 2 questions is yes then go to a new Talend web site, download Talend ESB v5, get to /examples/talend/tesb/locator-rest and proceed from there.

One thing I should mention is that Talend Open Studio for ESB is also available for download, it builds on top of Talend Data Integration suite and offers a second-to-none UI support for developing web services and routes. If you work with CXF SOAP Web Services or Camel then you probably won't be able to get back to your old UI tools after trying Talend Open Studio for ESB.

And the good news is that we are working on bringing the users a tooling for developing RESTful applications. It is a work in progress but it's happening, stay tuned !

Observations about Maven Central Search URIs

I was using a Maven Central Search Engine to find a Google GWT artifact. The search engine is very useful, no doubt about it. I then somehow got focused on the actual URI which identifies the specific GWT artifact I was after:

"http://search.maven.org#search|gav|1|g%3A'com.google.gwt'%20AND%20a%3A'gwt'".

I'm using single quotes just to keep the whole URI expression inside double quotes.

That does not seem like a user-friendly URI to me, and I guess it does not have to be, but I'd just like to analyze it a bit more.

I'm not sure what "|gav|1|" is about, but the main expression (encoded), is really this one:

"g%3A'com.google.gwt'%20AND%20a%3A'gwt'" which reads like this: "find resources with a group 'com.google.gwt' and artifact 'gwt'".

Now compare it with this equivalent FIQL expression which CXF will happily help to handle:
"g=='com.google.gwt';a=='gwt'"

thus giving us

"http://search.maven.org?_s=g=='com.google.gwt';a=='gwt'".

This query a bit simplistic but FIQL does start 'scaling' when we want to use composite queries, for example:

"http://search.maven.org?_s=g=='com.google.com';(a=='gwt',a=='gwt-inject');(v=gt=2.0;v=le=3.0)"

which reads "Find all the modules with a given group, with artifact set to 'gwt' or 'gwt-index' and having versions greater than 2.0 but less or equal to 3.0". It's impressive how FIQL can capture so complex queries in a compact way with a URI which humans can understand. FIQL simply rocks and I encourage users to experiment with it more. By the way, check out the client-side FIQL support, can be handy for building FIQL queries inside the HTML forms.

[OT] CXF- Even Better Than The Real Thing

Listening to U2 for a long time seems to start affecting me, so much that I can't stop 'producing' my own 'mix' (the Title line of this post) of the fantastic U2's "Even Better Than The Real Thing".
Those who follow this blog for a while will notice I'm getting a bit repetitive by mixing 'CXF' with phrases from some songs I've heard, sorry :-). But if you are working with CXF you may probably agree with the title :-). Have Fun.

Complex WADL Processing in CXF

We have seen a number of JIRA issues opened against a WadlToJava generator since it was added to CXF 2.4.1. Seeing the community experimenting with it was really reassuring and it confirmed there was some genuine interest toward working with WADL. WADL was a really fine effort from Marc Hadley and it does seem to have nearly all it needs to move from a submission to a final recommendation.

One of the major contributions to the CXF WadlToJava code generator came from Christos Fragoulides who did help to push it to the next level by providing a lot of feedback and fixes while working with a more up-to-date WADL document describing Rackspace Cloud Servers API. They have a complex WADL document which utilizes most of the WADL capabilities and it is in such cases where one can start seeing the advantages of being able to get a document and start testing the server in a matter of minutes. By the way, Rackspace, hope we can see a more up-do-date WADL document live soon :-).

Then, when I already thought that our WADL code generator was flying high, came another very good contribution from Lars Weber which should make it possible to get a useful code from WADL descriptions like this one. Now the generator supports local references for all the WADL elements, in addition to external resource references.

Note that the AtomPub example relies on a WADL link element which can be used to identify a portion of the representation. How one can get to a portion of the representation in CXF JAX-RS ? Easy, use an XPathProvider.

So please try the WADL code generator in CXF 2.5.0, stress it more and provide the valuable feedback.

Lets make OAuth go mainstream with CXF JAX-RS

OAuth 1.0 offers a fairly complex HTTP-centric solution to a classical problem of "how a resource owner (end user) can allow a third-party consumer to access some of the resources this user owns on the resource server".

I recall being quite confused about OAuth and what exactly it was supposed to help with, when starting to look into it about 2 years ago. I guess it was a typical reaction of someone trying to grasp a given technology without having any prior experience in the area. What was the story with that redirection, and then with yet another one, and how all the pieces were supposed to be wired together ? Was it about the authentication or authorization, and where would something like OpenId come in ? I was asking myself those kind of questions.

It takes a bit of time to realize how elegant OAuth actually is. OAuth 1.0 has been supported by major players such as Google, Facebook and many others. Jersey and RestEasy are offering their support. And OAuth 2.0 is going to make a massive impact real soon.

CXF 2.5.0 is joining the OAuth game. And we have tried our best to make it possible for users to write complete, functional, secure OAuth applications the way they write their HelloWorld demos.

Before providing more information, I'd like to thank Lukash Moren for providing a high-quality OAuth 1.0 implementation as part of his GSOC 2009 project, and Bill for asking me to work on OAuth during my brief spell at JBoss - it all kind of happened at the same time...

Now, please review this documentation providing a comprehensive overview of what it takes to create and deploy an OAuth server, with a lot of examples and hints on various approaches one may need to choose from when building a complete OAuth solution.

Much depends on how the resource server manages the user resources and provides an access the the end users and third-party consumers. Many options are available but the message we'd like to send is that you don't need to create a WEB application similar to those provided by Google or Facebook or Twitter in order to get working with OAuth. CXF JAX-RS provides the ingredients one may need to apply OAuth to all sort of problems requiring the end user authorizing a 3rd party access. Obviously we will be improving what is already there as time goes by.

So lets make OAuth go mainstream with CXF JAX-RS, start building OAuth applications and enjoy it all the way.

SAML Claims-Based Authorization for JAX-RS endpoints

SAML has been around for a while and is used in real-world applications. CXF WS-Security framework is already offering a second-to-none support for SAML, especially with the introduction of the CXF's own STS, please check Colm's and Oliver's blogs and have some exciting read if you work with WS. Actually, STS can become quite important even for the advanced RESTful security - so follow those blogs even you don't work with WS :-).

The good thing about SAML is that SAML-based solutions do not have to be WS-* based only. SAML is an extensible and rich language and all sort of SAML profiles can be utilized for RESTful applications too. Arguably SAML is complex but the fact it's being used around is critical.

CXF JAX-RS is 'keen' to help users start working with SAML. CXF 2.5.0 offers an initial support for including SAML assertions in the HTTP request payloads and using them to enforce the Claims-Based or legacy Role-Based Access Control rules on the deployed JAX-RS endpoints.

One can get a SAML Assertion posted to the server inside a wrapper XML element which will include the actual application payload such as Book and the assertion itself, with the complete envelope being signed only or with both the application payload and the assertion itself being signed individually. All of it is done by registering a couple of client and server side handlers which will transparently add an envelope, signatures and then validate it all on the server side with the application payload (Book) becoming a root XML node in the end, ready to be processed by the JAX-RS runtime.

With REST we do not always post the XML data so what you can also do is easily add an optionally deflated and then base64-encoded assertion either to the Authorization header or a form field. Such encoded values might also be signed - something we may support in due time.

Having a SAML assertion available as part of the application payload raises a question as to what can actually be done with it. SAML is often used to provide SSO solutions for RESTful services, but we are not there yet, so the best thing that we could come up with at this initial stage is to let JAX-RS users utilize SAML claims in order to enforce the access control rules.

SAML claims are much richer than those which can be asserted with the RBAC rules. However @RolesAllowed and friends are still used a lot and probably will be so it was important to make sure that the legacy rules can work with SAML assertions having claims which are actually representing roles and also offer the way to assert much richer rules which are possible with SAML claims.

Have a look please at this sample code fragment and the follow-up comments. One have a lot of flexibility in the way the claim-based rules can be expressed and also complement the existing RBAC rules without even modifying the code which can also be important.

Before such rules can be enforced, the assertion needs to be validated. The SAML handler will do its own validation, and if you have STS deployed them you may actually want to delegate to it to do some validation too (recall the advice at the start of this post :-)). See also how you can setup an STS client.

The question which has not been seriously addressed yet is how one will create SAML assertions in the first place. At this stage one can start experimenting with using a callback approach and I think we can offer a support for utilizing a WS STSClient on the client side for getting the assertions and transparently adding them to the payload easily enough - please note, the fact that STSClient is a WS client is the least important fact here, it's a perfect mediator between the client code and STS and it does not matter if a SAML assertion which will make it into a non-SOAP payload was actually fetched using a SOAP client.

Note that supporting other interesting SAML-based solutions is of interest to us, that involves the possibility of supporting a WEB Browser SSO profile, etc.

In meantime, please write your simple SAML callback implementation which will generate SAML assertions and start stressing your endpoints secured by claim-based rules, or even invent a new SAML profile along the way :-)

Stay tuned !

Simple XML Security with CXF JAX-RS

Enhancing the security support for JAX-RS services was a major theme during the CXF 2.5.0 development. CXF already offers a good support for users to create secure RESTful services relying on HTTPS and it also offers a number of useful utility classes for enforcing the authentication and authorization rules.

But the security can be a much richer subject, it is indeed and it is the right time now for us to start working on the advanced security features for CXF JAX-RS users to start experimenting and working with the message level security, tapping into the richness of SAML, deploying OAuth solutions. And we will be analyzing and providing a support for the most interesting and useful security features which are already being or will be used by the community.

As far as the message level security is concerned, XML Signature and XML Encryption are the two prominent W3C specifications which have been used as the basis for providing the message integrity and confidentiality without relying on HTTPS mainly for SOAP XML services, with the help of WS-Security related specifications.

In REST, XML is only one of the many formats which can be supported, but XML is still a very major format which is used a lot. Given the popularity of XML and also to simplify the integration with SOAP-based solutions, it does make sense to get the message-level security supported well for XML services and get to supporting other relevant efforts allowing to sign all sort of payloads next.

And this is what we did in CXF 2.5.0. We put a lot of effort in providing a solution that will make working with XML Security a child's play yet practical and able to deliver for users relying on JAX-RS.

XML Signature and XML Encryption are complex specifications but I hope you can agree after reading this section that working with these specifications and making them protect the messages can be interesting, simple and a real fun.

Note that all what is needed to get an XML signature applied to a given application payload is to register a single handler on client and server sides. By default, a signature will be enveloped as a last child inside a given XML instance such as Book. On the server side, the signature will be validated, removed from the payload and made available on the current Message for other handlers to use it if needed, for example, for optional SAML handlers be able to verify SubjectConfirmation methods such as "sender-vouches".

After the payload has become 'free' of its enclosed signature, it's wrapped in a useful CXF DOM-aware STAX reader and passed along to the JAX-RS runtime.

Making CXF JAX-RS produce enveloping or detached signatures is as easy as setting a signature style property on a client-side handler with the server side one capable of reading all types of signatures.

And now that we have a signed payload, isn't it tempting to get it encrypted too ? So here you go, all you need to get a self-contained EncryptedData with an embedded EncryptedKey capturing an encryption key is to register a single handler on client and server sides which will help CXF encrypt a payload (possibly signed) and then decrypt it on the server side.

One thing which is worth noting is that WSS4J is relied upon by this feature - this is an internal implementation detail and should be of no concern to users building advanced secure RESTful services. WSS4J has a lot of useful code and it makes sense at this stage to reuse it under the hood.

Give it a try please and as usual, please help us with the feedback. If you are a security expert - let us know what may need to be improved and if you are not - learn new advanced security concepts with CXF and become the one :-) Enjoy !

Capturing the info about roles with JAASLoginInterceptor

CXF ships a useful utility JAASLoginInterceptor which greatly simplifies the process of interacting with the JAAS subsystem and creating a current SecurityContext encapsulating the information about a user principal and roles which will be used for making authorization decisions. See this post for some more information.

When working with Apache Karaf, the simple way to get roles distinguished from a user principal is to configure the Karaf JAAS context such that role names start from a prefix such as "role_" and then let JAASLoginInterceptor know about it using a now deprecated "rolePrefix" property.

However when the existing stores containing the info about roles are used this simple technique may not work given that all sort of naming conventions can be used which may not 'managed' by a rolePrefix property. It makes it trickier to capture roles given that in Karaf the classes representing roles and user principals implement only a single Principal marker interface.

So in 2.5.0 what you can do in such cases is to use new roleClassifier and roleClassifierType properties to let JAASLoginInterceptor know how to get to the roles. Please see this section for an example (the updated content will be visible shortly). This enhancement came after the conversation with my Talend colleague Andrei Shakirin.

Additionally, Aki Yoshida enhanced the interceptor to better cope with the containers providing custom callbacks, for example, it will work perfectly with Jetty providing ObjectCallback instead of javax.security.auth.callback.PasswordCallback.

Finally note that SecurityContexts set by JAASLoginInterceptor on the current CXF message implement LoginSecurityContext. This one can be very handy for getting say Spring SecurityContexts populated, for doing the custom authorization based on the list of provided roles and using the underlying Subject returned from the JAAS subsystem as needed.

If you are asking at this stage, hmm, do I really need to continue using the servlet security, then you are most likely on the right track :-)

Enjoy!

Better support for big attachments in CXF 2.5.0

Dan had a high quality patch from Sam Meder applied just in time before CXF 2.5.0 has been released which is to do with limiting the max size of the incoming attachments - which is a very good improvement indeed. Whether you use WS and MTOM or depend on JAX-RS, a new "org.apache.cxf.io.CachedOutputStream.MaxSize" system property can be your friend, see this page for more info.
And virtually at the same time Dan did another fix for the runtime to cope better with attachments bigger than 2 GB, how about that :-) ?

It's great to see the CXF Runtime core delivering extremely well as far as working with attachments is concerned

All you need to get HTML views is RequestDispatcherProvider

While working on the new OAuth demo for the soon to be released Talend TSF distro based on Apache CXF 2.5.0 (see Dan's announcement), I relied a lot on RequestDispatcherProvider.

As you know a typical OAuth flow may involve a lot of interacting with the user and a complete demo should also show the user going to or being redirected to the 3rd party provider site, as well as approving the 3rd party accessing one or more of its private resources.

I was amazed, I really was, how simple and easy it was to get multiple RequestDispatcherProvider instances configured to redirect all sort of responses to JSP views handlers while still being able to return alternative representations such as XML ones.

I highly encourage people to try RequestDispatcherProvider, say goodbye to complex views integrations and never look back.

CXF Transform Feature and Redirection

The Transform feature is proving to be useful to CXF users, given that dynamically changing or dropping the incoming or outbound namespace(s) as well as updating the element names is often required when the changes have not been propagated across or are not even possible. The fact that all the modifications are done at the STAX level is critical as far as the performance is concerned.

The feature has been enhanced recently thanks to Aki Yoshida. The incoming payloads can get new elements added in a number of ways which can be very useful when the services validating the data using the close content schemas are in operation. Multiple updates (example, adding some elements, dropping another one, changing the name and or namespace of yet another one - all on the same payload) are better supported too.

There will be more enhancements coming in in time but I've been actually planning to highlight one of the lesser known tricks which one can use with the Transform feature, something that we demonstrate in our Talend distributions.

Consider the case where you have an endpoint deployed with hundreds or many thousands of clients consuming in. Those could be some long-running clients executing the code, possibly embedded in browsers, and that code is aware of the way this endpoint can be consumed. The time has come to deploy an updated endpoint. The more open the environment is - the fewer options are there to get the clients updated at the same time when the old endpoint goes down and the new one gets deployed.

This is not a new problem per se, but Transform Feature, in combination with the servlet redirection, offers its own simple way to tackle the cost of upgrading all the clients:

Keep the servlet serving the endpoint which is now down around but only have it redirecting all the requests from the old clients to the new servlet serving the new updated endpoint. This will keep the old clients happy for a while and the process of upgrading them can become less 'stressful'.

So now we will have a new endpoint serving the new clients but it will also get the requests redirected to it from the older clients. How will the new endpoint figure out how to handle a given request without resorting to a low-level XML or JSON manipulation ?

Yes, you are right - Transform Feature will help - it will ensure the old requests are recognized by the new endpoint and the responses from this new endpoint are recognized by the old clients still unaware of the fact they are talking to the new endpoint.

Here is how the relevant part of web.xml may look like:

<!-- Old Servlet -->
<servlet>
 <servlet-name>CXFServletV1</servlet-name>
 <display-name>CXFServletV1</display-name>
 <servlet-class>
   org.apache.cxf.transport.servlet.CXFServlet
 </servlet-class>
 <init-param>
  <param-name>redirect-servlet-path</param-name>
  <param-value>/v2</param-value>
 </init-param>
</servlet>

<!-- New Servlet -->
<servlet>
 <servlet-name>CXFServletV2</servlet-name>
 <display-name>CXFServletV2</display-name>
 <servlet-class>
   org.apache.cxf.transport.servlet.CXFServlet
 </servlet-class>
</servlet>

<servlet-mapping>
 <servlet-name>CXFServletV1</servlet-name>
 <url-pattern>/v1/*</url-pattern>
</servlet-mapping>

<servlet-mapping>
 <servlet-name>CXFServletV2</servlet-name>
 <url-pattern>/v2/*</url-pattern>
</servlet-mapping>

CXFServletV1 will redirect all the requests to the servlet listening on /v2 which is CXFServletV2.

Next we configure the Transform feature like this:

 
<bean id="transform" 
 class="org.apache.cxf.feature.StaxTransformFeature">
 <property name="contextPropertyName" 
              value="http.service.redirection"/>
 <!-- the rest of the feature config -->     
</bean>

The feature is configured to do the transformations only if a boolean property identified by the "contextPropertyName" is set on the current message. In this case, if the request has been redirected then the message will have an "http.service.redirection" set to true. It won't be the case for the requests coming from the new clients and thus the feature won't affect them.

Describing JSON Services in WADL

WADL can describe RESTful XML services really well, by having request and response representations linked to corresponding XML or Relax NG schema element declarations.
However, you've most likely seen the following fragments showing that both XML and JSON representations are supported:

<response>

<representation
   mediaType="application/xml"
   element="ns:Book""/>
<representation
   mediaType="application/json"/>
</response>

The above fragment can describe a method like this one:

@GET
@Produces(
{"application/xml",
 "application/json"})
public Book getBook() {...}

So given the WADL fragment above, the consumer can get an idea of how an XML response will look like, and thus how to consume and possibly validate it. The description of the JSON response representation gives no information at all.

That is not fair I can hear you, JSON fans, saying :-). Or perhaps I can hear people saying WADL is not good enough for describing JSON services. But CXF makes things possible :-), and hopefully the following two recent updates can help:

1. Setting a linkJsonToXmlSchema property on CXF WADLGenerator will result in a fragment like this one be created:

<response>

<representation
   mediaType="application/xml"
   element="ns:Book""/>
<representation
   mediaType="application/json"/>
   element="ns:Book""/>
</response>

Well, describing JSON representation with XML schema is not technically correct, but more often than not an XML schema element instance will give a pretty good idea how a JSON will look like, for example, a Book bean can be used to generate both XML and JSON sequences by the providers in charge.

Update: CXF WadlToJava generator has been updated to handle WADL representation elements with the "application/json" media type which link to XML Schema elements.

2. Setting HTTP Accept header to application/json when requesting WADL will result in a WADL JSON representation returned. For example, given

?_wadl&_type=json

will return a JSON-format WADL instance. Is that really cool ? You bet it is. What about WADL grammar referencing XML schemas, does it really make sense to JSON-format XML schemas ? No it does not, but we have a JSON schema initiative so what you need to do is just configure WADLGenerator to reference a schema like the one describing Products in this section and here you go. For example:

WADLGenerator wg = new WADLGenerator();
wg.setSchemaLocations(
   Collections.singletonList("json.schema"));
// do not create JAXB context
wg.setUseJaxbContextForQnames(false);
// let JSON provider handle it
wg.setIgnoreMessageWriters(false);

The provider can have the above properties set from Spring if needed.
The reference to the json.schema will be replaced with an absolute URI so the consumer handling a WADL JSON representation will be able to GET the referenced schema in the follow-up request.

For the above approach to become of some practical use, the tooling support is important. Consider opening a JIRA in the SoapUI project, vote for it and see what happens :-).

Enjoy !

What happened at Talend R&D summit

Last week many of us had a chance to attend to a 3 day R&D event organized by Talend in Paris. And what a summit it was.

It was good to meet Dan, JB, Christian, our colleagues from Talend Bonn's office and of course the hosts, people who organized and run the event. The only regret I have I could not play our regular chess game with Hadrian - but I guess we have plenty of time in the future to do it again :-)

I've had a chance to attend to similar events before and I did like it very much, but this last event stands on its own. Talend is still a young company but one can't help being impressed by what this team has already achieved. The tooling is very impressive, be it Talend ESB, core Talend Data Integration or Master Data Management UI suites. They understand what managing data, jobs, processes is about, what good UI is about.

Talend ESB team has had cool demos showing how ESB consumers and producers backed up by Apache CXF can be wired in together, run as live jobs, exported as OSGI bundles, with consumers locating endpoints and users seeing the management statistics. It is really impressive, I thought it was. The new Camel Builder is the tool to explore too - it also supports linking to Talend Jobs with each Job possibly representing a super complex chain on its own.

Talend team works very hard and this team knows how to play hard too :-).

Good to be part of Apache and Talend teams...

WADL First Development in CXF

The auto-generation of WADL documents has been supported in CXF JAX-RS for a while. Letting users get a WADL instance and use a tool like soapUI to test a live RESTful endpoint is very useful.

Don't get distracted by the fact WADL is not favoured by some REST advocates. Being able to test a live endpoint without spending time on writing a dedicated test code is good.

CXF JAX-RS also offers a CodeGenerator filter which can be configured on test/development/POC servers for possible users being able to download the source code, compile it and start experimenting immediately. Java is just the only option there for now but of course more languages can be supported easily.

That is all fine, but one limitation there is that what is possibly the most important WADL feature, underrated at the moment, is not really utilized. WADL is very resource centric and IMHO it's a perfect language for bridging would-be RESTful application modeling tools with the actual code. I'm not referring to UML-like modeling but a more high level approach with a tool linking entities representing root resources, subresources and data and using WADL as a internal link between a specific tool data representation and a wadl2java code generator. I'm not referring to a complex WADL XML editor here.

All/most of us do Java-first development and it's kind of easy to assume that nobody will need a document-first development. But the document-first approach has its followers as it has its benefits.

CXF is all about choice, options, diversity and letting users get their web services projects done using the strategies preferred in their teams. CXF 2.4.1 offers a server side WADL-first development support - give it a try and enjoy.

Talend joins JCP and JSR-339

This week Talend has joined Java Community Process (JCP) and JSR-339 (JAX-RS 2.0) Expert Group (EG) with myself acting as the company representative in this EG.

This is a good news for us working on Apache CXF as it's another vote of confidence from Talend in CXF continuing positioning itself as the best-of-breed web services framework for developing WS and REST compliant and robust service endpoints and consumers which just work.

Talend has a history of supporting open source projects and communities. Joining JCP and supporting JAX-RS 2.0 specification is another step in that direction, simply because JAX-RS 2.0 will become important for Apache CXF users building RESTful services.

Another reason is that Talend has a lot of experience in data-integration technologies. Talend tooling is powerful and impressive which will be a subject of another post. REST and data are good matches. Many interesting opportunities will arise for exposing data as RESTful endpoints, letting people search and link to the data of interest. It's natural Talend would like to get involved in JAX-RS 2.0.

See this blog entry for more information.

Failover support for CXF JAX-RS clients

You may recall all those discussions about "RESTful services can be consumed from browsers by humans only" statements. It was awhile back and of course it's been proven since then that it's possible to write a sophisticated client code consuming RESTful services in a number of ways.

In JAX-RS land, Paul Santoz innovated with introducing the fluent API with all/most JAX-RS stacks having custom implementations, and this API is now being standardized by JSR-339 expert group. Proxy based API is also supported by some stacks.

What is important to realize now is that JAX-RS client runtimes need to become more robust and sophisticated in order to move further, beyond supporting the assertion that "yes, we can do it, we can write the client code for working with RESTful services".

A human working with the browser has more space as far as time and decision making is concerned, when facing a connection failure for example. On the other hand, the code needs to be smarter and ready if the expectations that it does not exit immediately after a connection has become slow, broken or a target endpoint has been recycled is to be met.

This is where a failover feature comes in and it's been supported for CXF JAX-WS clients for a long time. Starting from CXF 2.4.1 it's also the case for CXF JAX-RS clients.

Please check this page for more info, experiment and provide the feedback.

It also confirms that in CXF we are committed to bringing the best support for developing SOAP and REST services. SOAP and REST developers have their 'differences' :-) but we understand them, bring both 'camps' together, and do our best to support them at the framework level.

As a side note, if you think about it, making clients failover-capable is a cheap way toward ensuring something close to 24-7 up time. If you are Amazon or BBC then you have all the hardware and experience in place to ensure the clients nearly never experience a "can't connect" problem. What if you are running an OSGI container hosting service endpoints and need to replace a service ?

Failover-enabling clients is a simple way toward making service endpoints replaceable without affecting the critical client code being executed somewhere. Configuring a CXF failover feature with a small timeout between retries will do the trick really well.

One thing which is really exciting for me is that my Talend colleagues are already working on providing enterprise-level failover-like features for CXF JAX-RS clients and I'm going to talk about it in one of the future posts.

If you have already integrated CXF JAX-RS in your higher-level product then I'd encourage you to follow Talend's lead.

Learn about major Apache projects with Talend

Talend Application Integration Division team is working on organizing 1 or 2 day courses designed to provide developers with free, hands-on training on Apache CXF, Apache Karaf, Apache Camel and Apache ActiveMQ projects.

Talend will organize the training days in locations where a reasonable number of people can attend.

Please take this 5 min survey and start looking forward to meeting Talend in your local area :)

How to use Camel transport with CXF JAX-RS endpoints

The following configuration shows how to have HTTP or JMS messages forwarded to
CXF JAX-RS endpoints in Camel:

<beans>
<jaxrs:server address="camel://direct:bookStore"
  serviceClass="service.BookStoreImpl"/>

<camelContext xmlns="http://camel.apache.org/schema/spring">
    
 <!-- JMS to CXF JAX-RS -->
 <route>
  <from uri="jms://test.bookStore"/>
  <to uri="direct:bookStore"/>
 </route>
        
 <!--  Jetty to CXF JAX-RS -->
 <route>
  <from 
uri="jetty:http://0.0.0.0:9002/bookstore?matchOnUriPrefix=true"/>
  <to uri="direct:bookStore"/>
 </route>
</camelContext>
</beans>

Note that you need to use "camel://direct:bookStore" as the address of the jaxrs endpoint and then use "direct:bookStore" in route definitions.

The above configuration will only work with the current CXF trunk and 2.3.x branch snapshots.
The address like "camel://direct:bookStore" is confusing CXF JAX-RS a bit in 2.3.4/2.4.0 and earlier.

However a workaround is available for earlier versions. Christian wrote a couple of Talend Integration Factory demos showing the workaround in action. It is just a matter of adding a simple handler into routes which will make sure CXF JAX-RS is capable of matching requests forwarded to "camel://direct:bookStore".

That said, I'd encourage those who are interested to check Christian's demos anyway, for example, the jaxrs-http-jms demo also shows how to build a Camel route which gets the response from the jaxrs endpoint and converts it using a JAXB converter and pushes it further to the JMS queue.

CXF JAX-RS: Moving ahead with WS-Trust integration

Making sure existing WS-Trust STS services can be used to enforce the security of deployed web services is of high importance to customers who have invested into WS-Trust based security solutions. It's just a common sense for them.

CXF provides some very advanced support for securing communications between JAX-WS consumers and providers and delegating to STS for validating the security tokens. CXF JAX-RS endpoints rely on HTTPS. That is as far as it can get at the moment. That is sufficient in many cases but the fundamental issue is that customers who want to use STS can not depend on it for additionally securing JAX-RS endpoints.

Now, I've seen people talking about RESTifying STS services, i.e, making sure plain HTTP consumers can talk to it. That is probably a worthy idea as it is likely will make it easier to communicate with STS. It does not solve the issue of using STS for validating all sort of security tokens (SAML assertions, etc) though.

But the thing is that in CXF we have a very good integration between JAX-WS and JAX-RS frontends.
Reusing JAX-WS and WS-Security runtimes for talking to STS is the fastest and most effective way to ensure JAX-RS endpoints can also be protected with STS.

Right now, we've done a very small step forward, we can get Basic Authentication credentials validated by STS, see this section for more info.

More work will have to be done for supporting SAML assertions passed via HTTPS or as part of XML security protected payloads.

OAuth and OpenId are also there. CAS is there. Other efforts are under way. These are of interest to us. We have a brilliant OAuth contribution from Łukasz Moreń in the CXF sandbox. But at this stage the integration with WS-Trust/STS is of high priority.

Stay tuned!

Transform Feature demonstrated in Talend Service Factory 2.4.0

Talend Service Factory 2.4.0 has several new advanced demos shipped as part of the examples distribution.

Please check Glen's blog for the information about WS-Trust and WS-SecurityPolicy demos.

Here I would like to describe a "jaxrs-jaxws-transformations" demo which shows the new Transform Feature at its best.

Many options for enhancing the existing XML schemas are possible. Usually, a namespace gets changed to indicate a breaking change to do with reordering elements, changing their names or structure or introducing new elements into a type with a closed schema content. Things are more complicated in the real world where best practices can require namespace changes for all updates made to a given schema.

The demo 'works' with the case where a Customer type has had a new optional property added to it. In addition, the namespace has also been modified, even though we are not dealing with the breaking update here.

The demo shows how:

* old and new JAX-RS clients can talk to existing (old) and new JAX-RS endpoints respectively - usual case
* old and new JAX-WS clients can talk to existing (old) and new JAX-WS endpoints respectively - usual case

Next, it shows how:

* new JAX-RS clients can talk to old JAX-RS endpoints
* new JAX-WS clients can talk to old JAX-WS endpoints

and

* old JAX-RS clients can talk to new JAX-RS endpoints
* old JAX-WS clients can talk to new JAX-WS endpoints

In this latter case, with old clients talking to new endpoints, two assumptions are made. First one is that the old clients know they are talking to new endpoints and thus Transform Feature affecting outgoing and incoming payloads is applied on the client side. Second one is that clients do not know that new endpoints will be used to handle their requests - the server side redirection and Transform Feature are used in tandem to make it work.

Finally, XMLSource is also shown in action on the client side.

This is a fairly involved demo trying to show how backward and forward compatibility may get achieved.

Try this demo and see if you can find it useful for the work you do in the real world.

Enjoy !

Building FIQL queries in CXF

Building FIQL queries is not a difficult process. Typical queries are simple enough to build them manually, for example:

_s=id=le=123

More complex queries relying on 'and' or 'or' conditions (possibly composed with each other) and multiple operators are a bit more tedious and complex to build.

CXF 2.4.0 introduces a SearchConditionBuilder utility class (thanks to Andy Michalec) which can be used to simplify building FIQL queries and working with CXF JAX-RS endpoints already supporting FIQL queries.

Have a look please at this section, try SearchConditionBuilder and provide the feedback.

Transforming XML in CXF

Working with XML based web services does not only involve having a proxy or HTTP client invoking on the remote endpoint with the underlying data-binding magically populating a given bean instance with the values contained in XML tags.

Working with XML also implies that some care has to be taken for all those clients distributed all over the enterprise and the WEB at large be able to consume the existing and newer endpoints.

If you have a newer endpoint deployed that can handle the payloads from the older clients then the backward compatibility is maintained. If the validation is enabled then this is usually achieved by adding optional elements to the updated schema.

What happens if you have so many endpoints that they have to be gradually replaced by newer ones, should newer clients be blocked from consuming the old endpoints ? Is it really needed if all the new payloads add is some ignorable content that makes the information about a given concept more complete for newer endpoints ? Is it even realistic as in the case of the WEB ? Is telling such clients 'wait, the endpoint is being upgraded' is the cheapest/simplest option ?

This is where the forward compatibility comes in and it implies that the unrecognized content has to be ignored. Disabling the validation is not always safe as the newer clients providing new content can expect that that new content has been processed into account as opposed to being ignored. Validation will prevent the important unrecognized tags from being dropped. But it also will prevent the forward compatibility altogether.

The new CXF Transform Feature has been introduced and hopefully it will help users with resolving all sorts of backward and forward compatibility issues. CXF JAX-WS and JAX-RS endpoints and clients can be configured for namespaces be dropped or changed, elements dropped, changed or appended to existing elements. This is all realized at the STAX XMLStreamReader and XMLStreamWriter levels so it is fast and effective.

Does your web service need to talk to the legacy server which does not understand what namespaces are ? Do you need to consume a response from the newer endpoint which has a new namespace introduced ? Do you need certain elements ignored or dropped on the input/output ?

Use the transformation feature, get inspired and write the services which just work and don't cause all the clients be recompiled/updated whenever a minor change to the new endpoint has been applied. Have the web service clients talking to a variety of endpoints without changing the code.

The use of the transform feature in combination with the servlet-based redirection allows for replacing the old endpoints with the new ones and redirecting the requests from the old clients to the new endpoints. Which is quite cool. See this web.xml (CXFServletV1 redirects to CXFServletV2, effectively changing final URI path from /v1/rest-transform to /v2/rest-transform), with CXFServletV2 serving a jaxrs:endpoint with the "restTransform" id, the last jaxrs endpoint in this beans.xml. Note this endpoint relies on the transform feature which drops the namespaces from the inbound payloads and changes the name of the outbound element, only for redirected requests - so that V1 clients can talk to V2 endpoints without V2 clients being affected.

Finally, if you use the default CXF JAX-RS JSONProvider then the transform feature can be applied to input/output JSON sequences too, surely you don't want the JSON clients failing to parse the JSON data if you happen to add one more property to the JAXB bean which is used to produce a JSON sequence :-).

Most of it can also be done by custom CXF interceptors as well, which can register custom STAX handlers or use XSLT or XPath. I've updated the Advanced XML section on the CXF JAX-RS wiki, have a look please.

Jettison 1.3 is released

As it happens, Dan and Dejan have let me manage the Jettison project from now on. And thus I'm happy to announce that Jettison 1.3 has been released.

Jettison 1.3 has had two patches from CXF users applied.

MappedXMLStreamReader can now read top-level explicit arrays containing qualified and unqualified tags - this allows JAXB to deserialize such JSON sequences into explicit collections (lists, etc). For this to work, the support for older convention assuming a top-level array may only contain a single node had to be dropped. Additionally, simple tags explicitly set to null have no CHARACTER events reported.

I think the good news is that Jettison will keep moving forward. One thing I'd like to clarify is that I won't be actively working with Jettison - however you can definitely expect patches addressing open JIRA issues being applied. Jettison has a small code base, so if you use it and see some issue then just check out the trunk and create a patch.

This latest release has really being driven by lingering issues opened against CXF JAX-RS but you don't have to use CXF for Jettison JIRAs be fixed - if you like working with Jettison then providing a patch is the only thing you need to do for the given issue be addressed :-)

Advanced Integration with the Talend Integration Factory

Talend Integration Factory, the Community Edition, has been released recently and is available for the download. Additionally, the Advanced Examples distribution can be downloaded from here.

The Talend Integration Factory (TIF) is an advanced platform for building real-world application integration solutions. It is centered around the powerful Apache Camel integration engine.

What I'm finding interesting about this solution is that the people involved into building it have the in-depth knowledge about what it takes to build the integration solutions which work and scale really well. Their optimism and the number of their ideas is boundless, and this solution will grow and deliver the real value for the customers.

Give the examples a try - the team is planning to bring you some really interesting and advanced examples in time, please see Christian's blog entry for more information.

JAX-RS Attachments demo in TSF 2.3.2

The Talend SF 2.3.2.0 distribution contains various minor improvements and has some dependencies, including the Karaf one, updated to new versions.

The Talend SF 2.3.2.0 Examples distribution contains a new jaxrs_attachments demo.

CXF JAX-RS offers a comprehensive support for reading and writing attachments and this demo tries to demonstrate it by showing how XOP and regular multiparts can easily be dealt with.

We are going to build upon it when we move on to doing the more advanced development with the proper support for multiparts being required.

Have a look at this demo if you're interested. And expect more interesting demos added soon. Stay tuned.

Unified Search Experience Made Easy With CXF

Providing the quality search experience is an important task for any serious web application project.

Most of the HTML forms let users do simple queries resulting in the equality or partial match checks on the server side. For example, "Find the book with the id starting from 123" or "Find all the books with the author's first name starting from Fred".

Imagine the following task. Provide a web interface which will let users search for all the books which have an id greater than 123. Or find the list if customers which paid less than 200 dollars.

What I've observed quite a few times is that when multiple services are built by individual teams within the same large organization or as part of single large project then every team will create its own query language.

One team will come with a custom query language built within this very team. And the other team working on some other service will build a slightly different variation of the custom query language.

The end result is that the users may need to learn two query languages, one in order to query the 1st service and the other one - in order to query the 2nd one.

Sometimes different teams will agree on using the same query language.

Using explicit SQL expressions is one option. Most likely a frontend UI tool will collect the user input and convert it into SQL and then pass it to the remote service. IMHO the use of SQL as a query language on the WEB should be discouraged for the obvious reasons: the fact the the end service uses an SQL database for storing the data is the very last thing the consumer should know about, just too much information is being leaked for this to work.

The use of XQuery or query languages created by Google Data and Microsoft teams is an entirely different approach. It does let developers provide a unified search experience to the users. For example, all the Google Data services have to support the same query language - something that users can appreciate.

As I mentioned in this post, CXF JAX-RS supports converting FIQL expressions into SearchCondition expressions which capture the FIQL queries and let users match them against the application data.

FIQL is indeed a simple language - please read this post from Arul for a nice introduction to FIQL. IMHO it does offer a viable alternative to more complex and advanced query languages and we'd like to continue enhancing the CXF search extensions for users be able to get the best out of FIQL.

The CXF SearchCondition interface offers a utility method for converting the FIQL queries to SQL expressions. This method (toSQL()) has been deprecated recently. While the users who find this method working for them may continue using it for a while, it is now recommended to use the SearchCondition visitors, thanks to Brian Topping for providing a patch.

It were possible to convert SearchCondition into more optimized SQL or non-SQL expressions even before the introduction of visitors but now the relevant code has become much cleaner. The SQLPrinterVisitor is shipped with CXF and it can be used to convert the queries to SQL, using the proper SQL aliases if needed. For example, imagine a query such as "a==b". The 'a' may easily be assumed to be the name of the column in some table - but we may not necessarily want the end users to 'hard code' the names of the columns in the queries; thus the SQL visitor lets the service developers to register an alias map, for the resulting query to contain say "A_Column" instead of 'a'.

I can imagine XQuery-aware and other visitors being added in time. In fact the way we are trying to build the search extensions is to make sure other query languages such as XQuery/etc are supported transparently. If users will start asking about supporting the new query language then we'll just provide the relevant SearchContext parser and SearchCondition visitor.

One immediate enhancement we are thinking of is to add a SearchQueryBuilder which users would use to build FIQL/etc queries using simple Java operations and pass the builder result to WebClients or proxies.

So imagine all the different web services within the same organization supporting the same simple URI friendly query language which is easy to understand and use. One thing you can be sure of is that the end users will appreciate it, especially when they start building their own client applications which need to query a number of those web services.

By the way, it should work nicely for CXF JAX-WS services too provided they've been JAX-RS-enabled.

JAX-WS and JAX-RS United in CXF

I've just reread the REST and SOAP United blog entry I did back in July 2008, more than 2 years ago.

This theme is as active as ever in CXF; as I've mentioned on this blog quite a few times we've been trying to provide the environment for SOAP developers to experiment with JAX-RS with as few changes as possible and indeed, quite a few CXF SOAP developers do use JAX-RS today while continuing running their SOAP services. Some developers are choosing to annotate the same service interface or implementation class with both JAX-WS and JAX-RS annotations, while some of those, preferring to do the WSDL-first approach, have tried to apply an external CXF JAX-RS user model, something that is demonstrated now in one of the Talend SF demos, and thus avoiding having to modify the (generated) code altogether.

There have been some new improvements recently in the integration between JAX-WS and JAX-RS runtimes.

First, many of the custom CXF annotations are now supported by the JAX-RS frontend. So one can easily do a combined JAX-WS and JAX-RS service which for example supports a FastInfoset feature by using CXF FastInfoset, Feature or In/OutInterceptors annotations, possibly also relying on say a DataBinding annotation enabling the CXF SDO DataBinding.

It was possible to wrap CXF DataBindings as JAX-RS MessageBodyReaders and MessageBodyWriters for a while, with XmlBeans being the latest DataBinding supported, thus effectively reusing the same provider instance between JAX-WS and JAX-RS endpoints.

But just a couple of weeks ago a CXF user asked about having JAX-RS MessageBodyWriters being used by SOAP JAX-WS endpoints. Does it sound a bit hairy to you :-), irrespectively of whether you are a JAX-WS or JAX-RS developer ?
It might sound so initially but I think it is just the next natural step in the way both frontends can get integrated.

A user who asked a question had developed an optimized XML Stax provider and did not want to rely on JAXB while working with JAX-WS services. So I just went ahead and added a utility JAX-RS DataBinding which can be plugged in into JAX-WS endpoints and which will delegate to custom JAX-RS writers and readers. I used a JAXBElementProvider shipped with the JAX-RS frontend to test this feature. Thus, even though most SOAP developers will unlikely use the feature due to a number of limitations, some of them might go ahead and see if some of the existing JAX-RS XML-aware providers can do something useful for their SOAP applications.

I've just thought about doing SOAP-Atom with Atom payloads being transmitted inside SOAP envelopes and handled by CXF JAX-RS Atom providers. But no, I won't advocate doing the double enveloping (soap:Envelope plus atom:feed) at all even though this feature makes it possible now :-).

Have fun :-)