Building FIQL queries is not a difficult process. Typical queries are simple enough to build them manually, for example:
_s=id=le=123
More complex queries relying on 'and' or 'or' conditions (possibly composed with each other) and multiple operators are a bit more tedious and complex to build.
CXF 2.4.0 introduces a SearchConditionBuilder utility class (thanks to Andy Michalec) which can be used to simplify building FIQL queries and working with CXF JAX-RS endpoints already supporting FIQL queries.
Have a look please at this section, try SearchConditionBuilder and provide the feedback.
Tuesday, April 19, 2011
Thursday, April 14, 2011
Transforming XML in CXF
Working with XML based web services does not only involve having a proxy or HTTP client invoking on the remote endpoint with the underlying data-binding magically populating a given bean instance with the values contained in XML tags.
Working with XML also implies that some care has to be taken for all those clients distributed all over the enterprise and the WEB at large be able to consume the existing and newer endpoints.
If you have a newer endpoint deployed that can handle the payloads from the older clients then the backward compatibility is maintained. If the validation is enabled then this is usually achieved by adding optional elements to the updated schema.
What happens if you have so many endpoints that they have to be gradually replaced by newer ones, should newer clients be blocked from consuming the old endpoints ? Is it really needed if all the new payloads add is some ignorable content that makes the information about a given concept more complete for newer endpoints ? Is it even realistic as in the case of the WEB ? Is telling such clients 'wait, the endpoint is being upgraded' is the cheapest/simplest option ?
This is where the forward compatibility comes in and it implies that the unrecognized content has to be ignored. Disabling the validation is not always safe as the newer clients providing new content can expect that that new content has been processed into account as opposed to being ignored. Validation will prevent the important unrecognized tags from being dropped. But it also will prevent the forward compatibility altogether.
The new CXF Transform Feature has been introduced and hopefully it will help users with resolving all sorts of backward and forward compatibility issues. CXF JAX-WS and JAX-RS endpoints and clients can be configured for namespaces be dropped or changed, elements dropped, changed or appended to existing elements. This is all realized at the STAX XMLStreamReader and XMLStreamWriter levels so it is fast and effective.
Does your web service need to talk to the legacy server which does not understand what namespaces are ? Do you need to consume a response from the newer endpoint which has a new namespace introduced ? Do you need certain elements ignored or dropped on the input/output ?
Use the transformation feature, get inspired and write the services which just work and don't cause all the clients be recompiled/updated whenever a minor change to the new endpoint has been applied. Have the web service clients talking to a variety of endpoints without changing the code.
The use of the transform feature in combination with the servlet-based redirection allows for replacing the old endpoints with the new ones and redirecting the requests from the old clients to the new endpoints. Which is quite cool. See this web.xml (CXFServletV1 redirects to CXFServletV2, effectively changing final URI path from /v1/rest-transform to /v2/rest-transform), with CXFServletV2 serving a jaxrs:endpoint with the "restTransform" id, the last jaxrs endpoint in this beans.xml. Note this endpoint relies on the transform feature which drops the namespaces from the inbound payloads and changes the name of the outbound element, only for redirected requests - so that V1 clients can talk to V2 endpoints without V2 clients being affected.
Finally, if you use the default CXF JAX-RS JSONProvider then the transform feature can be applied to input/output JSON sequences too, surely you don't want the JSON clients failing to parse the JSON data if you happen to add one more property to the JAXB bean which is used to produce a JSON sequence :-).
Most of it can also be done by custom CXF interceptors as well, which can register custom STAX handlers or use XSLT or XPath. I've updated the Advanced XML section on the CXF JAX-RS wiki, have a look please.
Working with XML also implies that some care has to be taken for all those clients distributed all over the enterprise and the WEB at large be able to consume the existing and newer endpoints.
If you have a newer endpoint deployed that can handle the payloads from the older clients then the backward compatibility is maintained. If the validation is enabled then this is usually achieved by adding optional elements to the updated schema.
What happens if you have so many endpoints that they have to be gradually replaced by newer ones, should newer clients be blocked from consuming the old endpoints ? Is it really needed if all the new payloads add is some ignorable content that makes the information about a given concept more complete for newer endpoints ? Is it even realistic as in the case of the WEB ? Is telling such clients 'wait, the endpoint is being upgraded' is the cheapest/simplest option ?
This is where the forward compatibility comes in and it implies that the unrecognized content has to be ignored. Disabling the validation is not always safe as the newer clients providing new content can expect that that new content has been processed into account as opposed to being ignored. Validation will prevent the important unrecognized tags from being dropped. But it also will prevent the forward compatibility altogether.
The new CXF Transform Feature has been introduced and hopefully it will help users with resolving all sorts of backward and forward compatibility issues. CXF JAX-WS and JAX-RS endpoints and clients can be configured for namespaces be dropped or changed, elements dropped, changed or appended to existing elements. This is all realized at the STAX XMLStreamReader and XMLStreamWriter levels so it is fast and effective.
Does your web service need to talk to the legacy server which does not understand what namespaces are ? Do you need to consume a response from the newer endpoint which has a new namespace introduced ? Do you need certain elements ignored or dropped on the input/output ?
Use the transformation feature, get inspired and write the services which just work and don't cause all the clients be recompiled/updated whenever a minor change to the new endpoint has been applied. Have the web service clients talking to a variety of endpoints without changing the code.
The use of the transform feature in combination with the servlet-based redirection allows for replacing the old endpoints with the new ones and redirecting the requests from the old clients to the new endpoints. Which is quite cool. See this web.xml (CXFServletV1 redirects to CXFServletV2, effectively changing final URI path from /v1/rest-transform to /v2/rest-transform), with CXFServletV2 serving a jaxrs:endpoint with the "restTransform" id, the last jaxrs endpoint in this beans.xml. Note this endpoint relies on the transform feature which drops the namespaces from the inbound payloads and changes the name of the outbound element, only for redirected requests - so that V1 clients can talk to V2 endpoints without V2 clients being affected.
Finally, if you use the default CXF JAX-RS JSONProvider then the transform feature can be applied to input/output JSON sequences too, surely you don't want the JSON clients failing to parse the JSON data if you happen to add one more property to the JAXB bean which is used to produce a JSON sequence :-).
Most of it can also be done by custom CXF interceptors as well, which can register custom STAX handlers or use XSLT or XPath. I've updated the Advanced XML section on the CXF JAX-RS wiki, have a look please.
Jettison 1.3 is released
As it happens, Dan and Dejan have let me manage the Jettison project from now on. And thus I'm happy to announce that Jettison 1.3 has been released.
Jettison 1.3 has had two patches from CXF users applied.
MappedXMLStreamReader can now read top-level explicit arrays containing qualified and unqualified tags - this allows JAXB to deserialize such JSON sequences into explicit collections (lists, etc). For this to work, the support for older convention assuming a top-level array may only contain a single node had to be dropped. Additionally, simple tags explicitly set to null have no CHARACTER events reported.
I think the good news is that Jettison will keep moving forward. One thing I'd like to clarify is that I won't be actively working with Jettison - however you can definitely expect patches addressing open JIRA issues being applied. Jettison has a small code base, so if you use it and see some issue then just check out the trunk and create a patch.
This latest release has really being driven by lingering issues opened against CXF JAX-RS but you don't have to use CXF for Jettison JIRAs be fixed - if you like working with Jettison then providing a patch is the only thing you need to do for the given issue be addressed :-)
Jettison 1.3 has had two patches from CXF users applied.
MappedXMLStreamReader can now read top-level explicit arrays containing qualified and unqualified tags - this allows JAXB to deserialize such JSON sequences into explicit collections (lists, etc). For this to work, the support for older convention assuming a top-level array may only contain a single node had to be dropped. Additionally, simple tags explicitly set to null have no CHARACTER events reported.
I think the good news is that Jettison will keep moving forward. One thing I'd like to clarify is that I won't be actively working with Jettison - however you can definitely expect patches addressing open JIRA issues being applied. Jettison has a small code base, so if you use it and see some issue then just check out the trunk and create a patch.
This latest release has really being driven by lingering issues opened against CXF JAX-RS but you don't have to use CXF for Jettison JIRAs be fixed - if you like working with Jettison then providing a patch is the only thing you need to do for the given issue be addressed :-)
Monday, February 21, 2011
Advanced Integration with the Talend Integration Factory
Talend Integration Factory, the Community Edition, has been released recently and is available for the download. Additionally, the Advanced Examples distribution can be downloaded from here.
The Talend Integration Factory (TIF) is an advanced platform for building real-world application integration solutions. It is centered around the powerful Apache Camel integration engine.
What I'm finding interesting about this solution is that the people involved into building it have the in-depth knowledge about what it takes to build the integration solutions which work and scale really well. Their optimism and the number of their ideas is boundless, and this solution will grow and deliver the real value for the customers.
Give the examples a try - the team is planning to bring you some really interesting and advanced examples in time, please see Christian's blog entry for more information.
The Talend Integration Factory (TIF) is an advanced platform for building real-world application integration solutions. It is centered around the powerful Apache Camel integration engine.
What I'm finding interesting about this solution is that the people involved into building it have the in-depth knowledge about what it takes to build the integration solutions which work and scale really well. Their optimism and the number of their ideas is boundless, and this solution will grow and deliver the real value for the customers.
Give the examples a try - the team is planning to bring you some really interesting and advanced examples in time, please see Christian's blog entry for more information.
Tuesday, February 1, 2011
JAX-RS Attachments demo in TSF 2.3.2
The Talend SF 2.3.2.0 distribution contains various minor improvements and has some dependencies, including the Karaf one, updated to new versions.
The Talend SF 2.3.2.0 Examples distribution contains a new jaxrs_attachments demo.
CXF JAX-RS offers a comprehensive support for reading and writing attachments and this demo tries to demonstrate it by showing how XOP and regular multiparts can easily be dealt with.
We are going to build upon it when we move on to doing the more advanced development with the proper support for multiparts being required.
Have a look at this demo if you're interested. And expect more interesting demos added soon. Stay tuned.
The Talend SF 2.3.2.0 Examples distribution contains a new jaxrs_attachments demo.
CXF JAX-RS offers a comprehensive support for reading and writing attachments and this demo tries to demonstrate it by showing how XOP and regular multiparts can easily be dealt with.
We are going to build upon it when we move on to doing the more advanced development with the proper support for multiparts being required.
Have a look at this demo if you're interested. And expect more interesting demos added soon. Stay tuned.
Friday, January 21, 2011
Unified Search Experience Made Easy With CXF
Providing the quality search experience is an important task for any serious web application project.
Most of the HTML forms let users do simple queries resulting in the equality or partial match checks on the server side. For example, "Find the book with the id starting from 123" or "Find all the books with the author's first name starting from Fred".
Imagine the following task. Provide a web interface which will let users search for all the books which have an id greater than 123. Or find the list if customers which paid less than 200 dollars.
What I've observed quite a few times is that when multiple services are built by individual teams within the same large organization or as part of single large project then every team will create its own query language.
One team will come with a custom query language built within this very team. And the other team working on some other service will build a slightly different variation of the custom query language.
The end result is that the users may need to learn two query languages, one in order to query the 1st service and the other one - in order to query the 2nd one.
Sometimes different teams will agree on using the same query language.
Using explicit SQL expressions is one option. Most likely a frontend UI tool will collect the user input and convert it into SQL and then pass it to the remote service. IMHO the use of SQL as a query language on the WEB should be discouraged for the obvious reasons: the fact the the end service uses an SQL database for storing the data is the very last thing the consumer should know about, just too much information is being leaked for this to work.
The use of XQuery or query languages created by Google Data and Microsoft teams is an entirely different approach. It does let developers provide a unified search experience to the users. For example, all the Google Data services have to support the same query language - something that users can appreciate.
As I mentioned in this post, CXF JAX-RS supports converting FIQL expressions into SearchCondition expressions which capture the FIQL queries and let users match them against the application data.
FIQL is indeed a simple language - please read this post from Arul for a nice introduction to FIQL. IMHO it does offer a viable alternative to more complex and advanced query languages and we'd like to continue enhancing the CXF search extensions for users be able to get the best out of FIQL.
The CXF SearchCondition interface offers a utility method for converting the FIQL queries to SQL expressions. This method (toSQL()) has been deprecated recently. While the users who find this method working for them may continue using it for a while, it is now recommended to use the SearchCondition visitors, thanks to Brian Topping for providing a patch.
It were possible to convert SearchCondition into more optimized SQL or non-SQL expressions even before the introduction of visitors but now the relevant code has become much cleaner. The SQLPrinterVisitor is shipped with CXF and it can be used to convert the queries to SQL, using the proper SQL aliases if needed. For example, imagine a query such as "a==b". The 'a' may easily be assumed to be the name of the column in some table - but we may not necessarily want the end users to 'hard code' the names of the columns in the queries; thus the SQL visitor lets the service developers to register an alias map, for the resulting query to contain say "A_Column" instead of 'a'.
I can imagine XQuery-aware and other visitors being added in time. In fact the way we are trying to build the search extensions is to make sure other query languages such as XQuery/etc are supported transparently. If users will start asking about supporting the new query language then we'll just provide the relevant SearchContext parser and SearchCondition visitor.
One immediate enhancement we are thinking of is to add a SearchQueryBuilder which users would use to build FIQL/etc queries using simple Java operations and pass the builder result to WebClients or proxies.
So imagine all the different web services within the same organization supporting the same simple URI friendly query language which is easy to understand and use. One thing you can be sure of is that the end users will appreciate it, especially when they start building their own client applications which need to query a number of those web services.
By the way, it should work nicely for CXF JAX-WS services too provided they've been JAX-RS-enabled.
Most of the HTML forms let users do simple queries resulting in the equality or partial match checks on the server side. For example, "Find the book with the id starting from 123" or "Find all the books with the author's first name starting from Fred".
Imagine the following task. Provide a web interface which will let users search for all the books which have an id greater than 123. Or find the list if customers which paid less than 200 dollars.
What I've observed quite a few times is that when multiple services are built by individual teams within the same large organization or as part of single large project then every team will create its own query language.
One team will come with a custom query language built within this very team. And the other team working on some other service will build a slightly different variation of the custom query language.
The end result is that the users may need to learn two query languages, one in order to query the 1st service and the other one - in order to query the 2nd one.
Sometimes different teams will agree on using the same query language.
Using explicit SQL expressions is one option. Most likely a frontend UI tool will collect the user input and convert it into SQL and then pass it to the remote service. IMHO the use of SQL as a query language on the WEB should be discouraged for the obvious reasons: the fact the the end service uses an SQL database for storing the data is the very last thing the consumer should know about, just too much information is being leaked for this to work.
The use of XQuery or query languages created by Google Data and Microsoft teams is an entirely different approach. It does let developers provide a unified search experience to the users. For example, all the Google Data services have to support the same query language - something that users can appreciate.
As I mentioned in this post, CXF JAX-RS supports converting FIQL expressions into SearchCondition expressions which capture the FIQL queries and let users match them against the application data.
FIQL is indeed a simple language - please read this post from Arul for a nice introduction to FIQL. IMHO it does offer a viable alternative to more complex and advanced query languages and we'd like to continue enhancing the CXF search extensions for users be able to get the best out of FIQL.
The CXF SearchCondition interface offers a utility method for converting the FIQL queries to SQL expressions. This method (toSQL()) has been deprecated recently. While the users who find this method working for them may continue using it for a while, it is now recommended to use the SearchCondition visitors, thanks to Brian Topping for providing a patch.
It were possible to convert SearchCondition into more optimized SQL or non-SQL expressions even before the introduction of visitors but now the relevant code has become much cleaner. The SQLPrinterVisitor is shipped with CXF and it can be used to convert the queries to SQL, using the proper SQL aliases if needed. For example, imagine a query such as "a==b". The 'a' may easily be assumed to be the name of the column in some table - but we may not necessarily want the end users to 'hard code' the names of the columns in the queries; thus the SQL visitor lets the service developers to register an alias map, for the resulting query to contain say "A_Column" instead of 'a'.
I can imagine XQuery-aware and other visitors being added in time. In fact the way we are trying to build the search extensions is to make sure other query languages such as XQuery/etc are supported transparently. If users will start asking about supporting the new query language then we'll just provide the relevant SearchContext parser and SearchCondition visitor.
One immediate enhancement we are thinking of is to add a SearchQueryBuilder which users would use to build FIQL/etc queries using simple Java operations and pass the builder result to WebClients or proxies.
So imagine all the different web services within the same organization supporting the same simple URI friendly query language which is easy to understand and use. One thing you can be sure of is that the end users will appreciate it, especially when they start building their own client applications which need to query a number of those web services.
By the way, it should work nicely for CXF JAX-WS services too provided they've been JAX-RS-enabled.
Sunday, January 9, 2011
JAX-WS and JAX-RS United in CXF
I've just reread the REST and SOAP United blog entry I did back in July 2008, more than 2 years ago.
This theme is as active as ever in CXF; as I've mentioned on this blog quite a few times we've been trying to provide the environment for SOAP developers to experiment with JAX-RS with as few changes as possible and indeed, quite a few CXF SOAP developers do use JAX-RS today while continuing running their SOAP services. Some developers are choosing to annotate the same service interface or implementation class with both JAX-WS and JAX-RS annotations, while some of those, preferring to do the WSDL-first approach, have tried to apply an external CXF JAX-RS user model, something that is demonstrated now in one of the Talend SF demos, and thus avoiding having to modify the (generated) code altogether.
There have been some new improvements recently in the integration between JAX-WS and JAX-RS runtimes.
First, many of the custom CXF annotations are now supported by the JAX-RS frontend. So one can easily do a combined JAX-WS and JAX-RS service which for example supports a FastInfoset feature by using CXF FastInfoset, Feature or In/OutInterceptors annotations, possibly also relying on say a DataBinding annotation enabling the CXF SDO DataBinding.
It was possible to wrap CXF DataBindings as JAX-RS MessageBodyReaders and MessageBodyWriters for a while, with XmlBeans being the latest DataBinding supported, thus effectively reusing the same provider instance between JAX-WS and JAX-RS endpoints.
But just a couple of weeks ago a CXF user asked about having JAX-RS MessageBodyWriters being used by SOAP JAX-WS endpoints. Does it sound a bit hairy to you :-), irrespectively of whether you are a JAX-WS or JAX-RS developer ?
It might sound so initially but I think it is just the next natural step in the way both frontends can get integrated.
A user who asked a question had developed an optimized XML Stax provider and did not want to rely on JAXB while working with JAX-WS services. So I just went ahead and added a utility JAX-RS DataBinding which can be plugged in into JAX-WS endpoints and which will delegate to custom JAX-RS writers and readers. I used a JAXBElementProvider shipped with the JAX-RS frontend to test this feature. Thus, even though most SOAP developers will unlikely use the feature due to a number of limitations, some of them might go ahead and see if some of the existing JAX-RS XML-aware providers can do something useful for their SOAP applications.
I've just thought about doing SOAP-Atom with Atom payloads being transmitted inside SOAP envelopes and handled by CXF JAX-RS Atom providers. But no, I won't advocate doing the double enveloping (soap:Envelope plus atom:feed) at all even though this feature makes it possible now :-).
Have fun :-)
This theme is as active as ever in CXF; as I've mentioned on this blog quite a few times we've been trying to provide the environment for SOAP developers to experiment with JAX-RS with as few changes as possible and indeed, quite a few CXF SOAP developers do use JAX-RS today while continuing running their SOAP services. Some developers are choosing to annotate the same service interface or implementation class with both JAX-WS and JAX-RS annotations, while some of those, preferring to do the WSDL-first approach, have tried to apply an external CXF JAX-RS user model, something that is demonstrated now in one of the Talend SF demos, and thus avoiding having to modify the (generated) code altogether.
There have been some new improvements recently in the integration between JAX-WS and JAX-RS runtimes.
First, many of the custom CXF annotations are now supported by the JAX-RS frontend. So one can easily do a combined JAX-WS and JAX-RS service which for example supports a FastInfoset feature by using CXF FastInfoset, Feature or In/OutInterceptors annotations, possibly also relying on say a DataBinding annotation enabling the CXF SDO DataBinding.
It was possible to wrap CXF DataBindings as JAX-RS MessageBodyReaders and MessageBodyWriters for a while, with XmlBeans being the latest DataBinding supported, thus effectively reusing the same provider instance between JAX-WS and JAX-RS endpoints.
But just a couple of weeks ago a CXF user asked about having JAX-RS MessageBodyWriters being used by SOAP JAX-WS endpoints. Does it sound a bit hairy to you :-), irrespectively of whether you are a JAX-WS or JAX-RS developer ?
It might sound so initially but I think it is just the next natural step in the way both frontends can get integrated.
A user who asked a question had developed an optimized XML Stax provider and did not want to rely on JAXB while working with JAX-WS services. So I just went ahead and added a utility JAX-RS DataBinding which can be plugged in into JAX-WS endpoints and which will delegate to custom JAX-RS writers and readers. I used a JAXBElementProvider shipped with the JAX-RS frontend to test this feature. Thus, even though most SOAP developers will unlikely use the feature due to a number of limitations, some of them might go ahead and see if some of the existing JAX-RS XML-aware providers can do something useful for their SOAP applications.
I've just thought about doing SOAP-Atom with Atom payloads being transmitted inside SOAP envelopes and handled by CXF JAX-RS Atom providers. But no, I won't advocate doing the double enveloping (soap:Envelope plus atom:feed) at all even though this feature makes it possible now :-).
Have fun :-)
Subscribe to:
Posts (Atom)
