JAX magazine for Java developers features articles and tutorials which can help to get the most of all the new and cool technologies and developments happening in the Java land today.
The latest issue offers, among other features, the tutorial introducing Feed Item Query Language (FIQL) and how it is currently supported in Apache CXF. Please download the issue as a PDF or get it over your preferred channel (on iTunes, etc), and see what you can do with FIQL - and provide the valuable feedback.
Enjoy!
Monday, February 18, 2013
Tuesday, January 29, 2013
Jettison 1.3.3 is out
Jettison 1.3.3 has been released earlier this month, please check the Download page.
This release ships a better MappedXMLStreamReader: arrays to be mapped to primitive collections in JAXB beans can be correctly processed without the loss of the data, some if its methods dealing with text will work properly as expected and the callers can identify an absolute position of the first illegal JSON character after XMLStreamException has been thrown.
As usual, I'd like to thank the users who have reported the issues against Jettison and would like to encourage to get the patches provided.
I think over the time it has become clearer to me why Jettison is still being used, at least by some of CXF users, even though such powerful and popular processors as Jackson exist. Jettison is a very lightweight library, it has just few basic STAX XMLStreamReader and XMLStreamWriter implementations: this alone is not sufficient to produce a perfect JSON. But in CXF at least Jettison can be customized a lot to make it produce if not perfect but close enough JSON sequences.
This is what keeps it visible really in CXF land at least IMHO - the fact that the users can use it to write or read simple sequences (with many JSON consumers actually expecting simple enough sequences) but also nearly completely control the way a given sequence is produced or consumed. For example, people are talking about backward and forward compatibility with respect to processing XML - same can be achieved for JSON in CXF, all without code-related changes.
This is where its niche is, a bridge between JAXB and JSON, simple to medium complexity sequences, the possibility of the easy enough customization.
If you work with Jettison: please test the latest release and provide the feedback, thanks
This release ships a better MappedXMLStreamReader: arrays to be mapped to primitive collections in JAXB beans can be correctly processed without the loss of the data, some if its methods dealing with text will work properly as expected and the callers can identify an absolute position of the first illegal JSON character after XMLStreamException has been thrown.
As usual, I'd like to thank the users who have reported the issues against Jettison and would like to encourage to get the patches provided.
I think over the time it has become clearer to me why Jettison is still being used, at least by some of CXF users, even though such powerful and popular processors as Jackson exist. Jettison is a very lightweight library, it has just few basic STAX XMLStreamReader and XMLStreamWriter implementations: this alone is not sufficient to produce a perfect JSON. But in CXF at least Jettison can be customized a lot to make it produce if not perfect but close enough JSON sequences.
This is what keeps it visible really in CXF land at least IMHO - the fact that the users can use it to write or read simple sequences (with many JSON consumers actually expecting simple enough sequences) but also nearly completely control the way a given sequence is produced or consumed. For example, people are talking about backward and forward compatibility with respect to processing XML - same can be achieved for JSON in CXF, all without code-related changes.
This is where its niche is, a bridge between JAXB and JSON, simple to medium complexity sequences, the possibility of the easy enough customization.
If you work with Jettison: please test the latest release and provide the feedback, thanks
Friday, December 28, 2012
Say Goodbye to HTTP URI Query Parameters
New Year is approaching fast but there is still some time to make one more New Year resolution.
Traditionally, when it comes to expressing the search requirements with HTTP URI, one uses URI query name and value, for example:
1. "/search/people?age=30&age=40"
2. "/search/people?ageFrom=30&ageTill=40"
First query can be read like this: "Find all people who are either 30 or 40 years old", the second - "Find all people older than 30 but younger than 40".
Plain query parameters are very widely used and actually very 'capable', one can invent a query parameter such as "ageFrom" to indicate the conditional requirement. It is not quite perfect in that the actual operator is "=" but the name implies it is not "equal to" but "greater or equal to", but it works, and probably reads better in some cases.
That said, the bigger the number of terms, the more tedious it can become to support creating the parameters like "ageFrom" and writing the actual code for figuring out which parameter means whether it is indeed the equality check as in "age=30" or say a "greater than" check as in "ageFrom=30". Supporting the finer-level checks such as "greater or equal to", as well as combining multiple type of checks can indeed become difficult to manage at the code level.
Consider always using FIQL expressions:
1. "/search/people?_s=age==30;age=40"
1.1 "/search/people?age==30;age=40"
1.2 "/search/people/age==30;age=40"
2. "/search/people?_s=age=ge=30;age=lt=40"
2.1 "/search/people?age=ge=30;age=lt=40"
2.2 "/search/people/age=ge=30;age=lt=40"
Does it look more complex ? I don't think it does but I can agree it may look a bit unusual at first, though the syntax is actually very easy to understand after typing it just for a bit of time.
Note a number of possible variations - in fact I guess I like options 1.2 and 2.2 most, the "optionality" aspect of URI query components does not make the whole URI represent a specific application state very cleanly (options 1, 1.1, 2, 2.1).
Also note the fine-level checks in 2.1/2.2 which are tricky to express with the plain queries, example, in this case it is "Find all people 30 years old or older but younger than 40 (exclusive)".
If you think about it, you have nothing to lose by starting using FIQL, for many cases it is nearly as simple and primitive as using plain query name and value pairs but the bonus comes immediately once you start expressing something more involved. As noted at the beginning of this post, plain queries are also capable in this regard - but the more complex the search requirements are - the less easy it becomes working with the plain queries.
Those Apache CXF users who have started experimenting with FIQL can utilize the existing FIQL converters, which is another plus point, consider the task of writing the code for converting plain query parameters to JPA2 expressions - it can be an involved task indeed - but with CXF JPA2 converter it is a walk in the park; as a side note - JPA2 converter now supports "count" extensions, for example, "find all the people living in London with more than 6 children".
Well, you may not be convinced that it is time to drop plain query name and value pairs. Never mind, just get CXF converting them internally to FIQL (which is very easy to do for CXF given that FIQL is a richer language, and still rely on the handy FIQL converters of your choice to interact with the data stores and completely generalize the search processing logic along the way. Offer different search endpoints for your users to enjoy working with, one accepting the traditional queries and another one accepting FIQL queries, with both endpoints using the same FIQL converters.
Still plenty of options to innovate in this well-explored space :-)!
Update: See also this earlier post from Abhishek Jain.
Happy New Year !
Traditionally, when it comes to expressing the search requirements with HTTP URI, one uses URI query name and value, for example:
1. "/search/people?age=30&age=40"
2. "/search/people?ageFrom=30&ageTill=40"
First query can be read like this: "Find all people who are either 30 or 40 years old", the second - "Find all people older than 30 but younger than 40".
Plain query parameters are very widely used and actually very 'capable', one can invent a query parameter such as "ageFrom" to indicate the conditional requirement. It is not quite perfect in that the actual operator is "=" but the name implies it is not "equal to" but "greater or equal to", but it works, and probably reads better in some cases.
That said, the bigger the number of terms, the more tedious it can become to support creating the parameters like "ageFrom" and writing the actual code for figuring out which parameter means whether it is indeed the equality check as in "age=30" or say a "greater than" check as in "ageFrom=30". Supporting the finer-level checks such as "greater or equal to", as well as combining multiple type of checks can indeed become difficult to manage at the code level.
Consider always using FIQL expressions:
1. "/search/people?_s=age==30;age=40"
1.1 "/search/people?age==30;age=40"
1.2 "/search/people/age==30;age=40"
2. "/search/people?_s=age=ge=30;age=lt=40"
2.1 "/search/people?age=ge=30;age=lt=40"
2.2 "/search/people/age=ge=30;age=lt=40"
Does it look more complex ? I don't think it does but I can agree it may look a bit unusual at first, though the syntax is actually very easy to understand after typing it just for a bit of time.
Note a number of possible variations - in fact I guess I like options 1.2 and 2.2 most, the "optionality" aspect of URI query components does not make the whole URI represent a specific application state very cleanly (options 1, 1.1, 2, 2.1).
Also note the fine-level checks in 2.1/2.2 which are tricky to express with the plain queries, example, in this case it is "Find all people 30 years old or older but younger than 40 (exclusive)".
If you think about it, you have nothing to lose by starting using FIQL, for many cases it is nearly as simple and primitive as using plain query name and value pairs but the bonus comes immediately once you start expressing something more involved. As noted at the beginning of this post, plain queries are also capable in this regard - but the more complex the search requirements are - the less easy it becomes working with the plain queries.
Those Apache CXF users who have started experimenting with FIQL can utilize the existing FIQL converters, which is another plus point, consider the task of writing the code for converting plain query parameters to JPA2 expressions - it can be an involved task indeed - but with CXF JPA2 converter it is a walk in the park; as a side note - JPA2 converter now supports "count" extensions, for example, "find all the people living in London with more than 6 children".
Well, you may not be convinced that it is time to drop plain query name and value pairs. Never mind, just get CXF converting them internally to FIQL (which is very easy to do for CXF given that FIQL is a richer language, and still rely on the handy FIQL converters of your choice to interact with the data stores and completely generalize the search processing logic along the way. Offer different search endpoints for your users to enjoy working with, one accepting the traditional queries and another one accepting FIQL queries, with both endpoints using the same FIQL converters.
Still plenty of options to innovate in this well-explored space :-)!
Update: See also this earlier post from Abhishek Jain.
Happy New Year !
Wednesday, December 12, 2012
FIQL and JPA2 Queries In Action
I've been focusing quite a lot recently on enhancing CXF Search extension module, by improving the existing converters and creating the new ones, making sure the parser is configurable, flexible and capable of mapping arbitrary property names to the properties of the bean capturing the search expression, and improving the documentation.
Andy Michalec created a FIQL parser quite a long time ago, it's been around for a while really, but it is only since Jeff Wang provided an initial FIQL to JPA2 converter patch that it kind of struck me how important it was to ensure the utility converters for mainstream and most popular technologies for querying the data stores were available.
With all the documentation and code improvements, it is still not quite good till it is actually demonstrated somehow, and this is exactly what I've spent the last few days upon, on enhancing the existing Talend ESB jaxrs-advanced demo to show how the client can type the queries with SearchConditionBuilder without having to type FIQL expressions manually and get the expressions seamlessly converted to JPA2 TypedQuery and CriteriaQuery, with Tuple mixed in, and the matching data returned to the client, all with the help of FIQL to JPA2 converter.
Let me comment on the actual demo enhancements in more detail.
First check the client code, the "useSearchService" function, starting from the line 194 and more specifically from line 205. IMHO expressing the query with the help of the fluent query builder (also created by Andy) is quite cool, it is definitely more descriptive with respect to what is actually expected, and it is the specific query language agnostic, it is FIQL by default but can be something else.
The builder can build whatever advanced query expression is needed, though practically speaking I would not expect massive expressions being created.
Now lets move to the server code. SearchService is one which actually handles the client requests and all it does it delegates to PersonInfoStorage which deals with querying the data. Note, SearchService works with CXF SearchContext by expecting it to extract the search expression from the current URI query component (default mode) or getting the expression from URI path and submitting it to SearchContext.
The main and in fact the single demo domain entity is Person but it is the one which is recursive, with children and parents, ancestors and descendants linked to. I haven't modified these relationships to get CXF JPA2 converter working properly, the only thing I did I added a couple of missing Person setters, added JPA2 annotations and also a hack required for JAXB RI capable of dealing with recursive structures (see the end of Person code).
The Person model is initialized in PersonInfoStorage init method (see the end of the file), where the injected JPA2 EntityManager is used - it could've been done elsewhere but was good enough for me for the purpose of the demo.
Next check getTypedQueryPerson (line 68) - see how straightforward it is to get the FIQL expression transparently converted to JPA2 TypedQuery, the comments in the code should make it very easy to understand.
Note, Person is a recursive and possibly a very deep structure and one may not necessarily want to fetch all the Person representation back to the client. A number of options exists for dealing with this issue including using the intrusive JAXB XmlTransient annotation but using JPA2 Tuple is one of the most elegant ones.
Using Tuple is one of JPA2 options for having the response shaped into simpler or different structures and getCriteriaQueryPerson method shows how it can be easily done by having the query returning the data sufficient for initializing a simple PersonInfo representation. One needs to get OpenJPA auto-generating the
metamodel classes for the tuple selections working and you can see how it can be done here, this tip helped me a lot.
Also note that PersonInfoStorage has the injected bean properties passed to SearchContext. Why this may have to be done is explained here but in short it lets to get the property names used in query language completely decoupled from the corresponding properties of the capturing bean, for example, in the demo case, the client can type "builder.is('childName').equalTo('Fred')" and have it working with the 'childName' correctly mapped to "children.name", where "children" points to a Person collection of children. This makes it simpler and easier to work with for the client, while keeping the internals of how the properties are actually linked to each other completely opaque. That is cool, I can hear you saying :-)
Finally have a look at how JPA2 EntityManager is initialized and injected. I started with arguably a simpler approach, I basically used Spring ORM module to get the entity manager wired in, see this beans.xml. This actually back-fired on me when I tried to make the demo working in Karaf - Spring ORM needs to see persistence.xml and is not capable on its own of inspecting OSGI Meta-Persistence property set up in the common module's pom.xml so I just ended up duplicating persistence.xml in both common and service modules.
On the TODO list is to follow an excellent tutorial from Christian and make the demo working with OSGi JPA Service. Perhaps someone from the community will be interested in doing a related pull request ?
So this is it and hope you'll find something interesting in this demo, enjoy !
Andy Michalec created a FIQL parser quite a long time ago, it's been around for a while really, but it is only since Jeff Wang provided an initial FIQL to JPA2 converter patch that it kind of struck me how important it was to ensure the utility converters for mainstream and most popular technologies for querying the data stores were available.
With all the documentation and code improvements, it is still not quite good till it is actually demonstrated somehow, and this is exactly what I've spent the last few days upon, on enhancing the existing Talend ESB jaxrs-advanced demo to show how the client can type the queries with SearchConditionBuilder without having to type FIQL expressions manually and get the expressions seamlessly converted to JPA2 TypedQuery and CriteriaQuery, with Tuple mixed in, and the matching data returned to the client, all with the help of FIQL to JPA2 converter.
Let me comment on the actual demo enhancements in more detail.
First check the client code, the "useSearchService" function, starting from the line 194 and more specifically from line 205. IMHO expressing the query with the help of the fluent query builder (also created by Andy) is quite cool, it is definitely more descriptive with respect to what is actually expected, and it is the specific query language agnostic, it is FIQL by default but can be something else.
The builder can build whatever advanced query expression is needed, though practically speaking I would not expect massive expressions being created.
Now lets move to the server code. SearchService is one which actually handles the client requests and all it does it delegates to PersonInfoStorage which deals with querying the data. Note, SearchService works with CXF SearchContext by expecting it to extract the search expression from the current URI query component (default mode) or getting the expression from URI path and submitting it to SearchContext.
The main and in fact the single demo domain entity is Person but it is the one which is recursive, with children and parents, ancestors and descendants linked to. I haven't modified these relationships to get CXF JPA2 converter working properly, the only thing I did I added a couple of missing Person setters, added JPA2 annotations and also a hack required for JAXB RI capable of dealing with recursive structures (see the end of Person code).
The Person model is initialized in PersonInfoStorage init method (see the end of the file), where the injected JPA2 EntityManager is used - it could've been done elsewhere but was good enough for me for the purpose of the demo.
Next check getTypedQueryPerson (line 68) - see how straightforward it is to get the FIQL expression transparently converted to JPA2 TypedQuery, the comments in the code should make it very easy to understand.
Note, Person is a recursive and possibly a very deep structure and one may not necessarily want to fetch all the Person representation back to the client. A number of options exists for dealing with this issue including using the intrusive JAXB XmlTransient annotation but using JPA2 Tuple is one of the most elegant ones.
Using Tuple is one of JPA2 options for having the response shaped into simpler or different structures and getCriteriaQueryPerson method shows how it can be easily done by having the query returning the data sufficient for initializing a simple PersonInfo representation. One needs to get OpenJPA auto-generating the
metamodel classes for the tuple selections working and you can see how it can be done here, this tip helped me a lot.
Also note that PersonInfoStorage has the injected bean properties passed to SearchContext. Why this may have to be done is explained here but in short it lets to get the property names used in query language completely decoupled from the corresponding properties of the capturing bean, for example, in the demo case, the client can type "builder.is('childName').equalTo('Fred')" and have it working with the 'childName' correctly mapped to "children.name", where "children" points to a Person collection of children. This makes it simpler and easier to work with for the client, while keeping the internals of how the properties are actually linked to each other completely opaque. That is cool, I can hear you saying :-)
Finally have a look at how JPA2 EntityManager is initialized and injected. I started with arguably a simpler approach, I basically used Spring ORM module to get the entity manager wired in, see this beans.xml. This actually back-fired on me when I tried to make the demo working in Karaf - Spring ORM needs to see persistence.xml and is not capable on its own of inspecting OSGI Meta-Persistence property set up in the common module's pom.xml so I just ended up duplicating persistence.xml in both common and service modules.
On the TODO list is to follow an excellent tutorial from Christian and make the demo working with OSGi JPA Service. Perhaps someone from the community will be interested in doing a related pull request ?
So this is it and hope you'll find something interesting in this demo, enjoy !
Friday, November 30, 2012
Use FIQL to query LDAP and OSGI containers
It has taken me about 90 minutes to write an initial FIQL to LDAP converter which outputs the query formatted according to RFC 4515 and document it here.
I have copied most of the boiler-plate code from the FIQL to SQL converter and given how simple the LDAP query is it was very easy to finish it off fast. The point is that other custom converters (example FIQL to CQL) can likely be written even faster by copying and pasting the LDAP converter :-).
More work will likely need to be done to get RFC-4515 supported completely, but the converter should already be capable to support many simple or composite LDAP queries which do not use complex characters, etc.
You may want to ask why would not one support LDAP query language by directly encoding it into request URIs. There are two issues here, first, the LDAP language is not quite URI friendly, the second one is that it will leak the details about the data technology supporting a given REST endpoint.
This is why FIQL or other dedicated language like OData is the way to go for doing medium complexity queries over HTTP. It is easy to read and it lets the users completely encapsulate the data source details.
I think with this latest update CXF Search extension has enough 'material' for users to start experimenting with using FIQL on top of the well-known and traditional data sources.
I can nearly see some of you starting thinking of creating some cool UI search module which will let users do more interesting search queries over the containers such as LDAP stores or OSGI runtimes with this new converter and may be even sharing the tested queries with the other users who may find it useful.
The next phase is to get FIQL a bit closer to all those No SQL data stores and it should be quite interesting.
Watch this space :-)
I have copied most of the boiler-plate code from the FIQL to SQL converter and given how simple the LDAP query is it was very easy to finish it off fast. The point is that other custom converters (example FIQL to CQL) can likely be written even faster by copying and pasting the LDAP converter :-).
More work will likely need to be done to get RFC-4515 supported completely, but the converter should already be capable to support many simple or composite LDAP queries which do not use complex characters, etc.
You may want to ask why would not one support LDAP query language by directly encoding it into request URIs. There are two issues here, first, the LDAP language is not quite URI friendly, the second one is that it will leak the details about the data technology supporting a given REST endpoint.
This is why FIQL or other dedicated language like OData is the way to go for doing medium complexity queries over HTTP. It is easy to read and it lets the users completely encapsulate the data source details.
I think with this latest update CXF Search extension has enough 'material' for users to start experimenting with using FIQL on top of the well-known and traditional data sources.
I can nearly see some of you starting thinking of creating some cool UI search module which will let users do more interesting search queries over the containers such as LDAP stores or OSGI runtimes with this new converter and may be even sharing the tested queries with the other users who may find it useful.
The next phase is to get FIQL a bit closer to all those No SQL data stores and it should be quite interesting.
Watch this space :-)
Thursday, November 29, 2012
[OT] Just Say Yes to CXF
This is a regular, once or so per year, totally off-topic post dedicated to linking the music I listen to to CXF :-)
The New Year is coming soon, time to start thinking about the next year, about the decisions which will make the professional life of developers working with web services front-ends something to really look forward to.
It has never been easy to choose which framework to use to get web services up and running. It is tough. At a decision time like this one, when it is cold outside, it is good to listen to Snow Patrol, which is one of my favorite bands.
Some of their music is absolutely brilliant. Listen to it, think about it, and Just Say Yes to CXF.
I've had some fun writing this post :-), hope you'll have some fun too !
The New Year is coming soon, time to start thinking about the next year, about the decisions which will make the professional life of developers working with web services front-ends something to really look forward to.
It has never been easy to choose which framework to use to get web services up and running. It is tough. At a decision time like this one, when it is cold outside, it is good to listen to Snow Patrol, which is one of my favorite bands.
Some of their music is absolutely brilliant. Listen to it, think about it, and Just Say Yes to CXF.
I've had some fun writing this post :-), hope you'll have some fun too !
Thursday, November 22, 2012
How to refresh OAuth2 access tokens in CXF
OAuth2 Refresh Token grant lets OAuth2 clients owning an access token refresh it with a new access token if the current attempt to access the end user's resources has failed.
Refresh tokens offer an advanced support for the OAuth2-protected applications to force the clients to re-authenticate regularly without forcing them to go via the re-authorization step involving the end user which may not always be practical.
They can also help with effectively getting the original access tokens revoked and replaced with the modified access tokens, example, with the new opaque scopes limiting or extending the current client permissions or even replacing the token type itself, example, replacing Bearer with MAC, all dynamically.
During the refresh process, the actual refresh token may also be 'renewed' with the new value by having the so-called key-rotation scheme implemented.
One possible approach is to have a short-lived access token and long-lived refresh token. Whenever the access token expires the client is forced to refresh with a valid refresh token and re-authenticate along the way.
I'd like to encourage developers to check OAuth2 archives on some interesting insights from the security experts on why using refresh tokens might benefit a particular OAuth2 application.
All this sounds good but you may want to ask, how to actually work with the refresh token in practice, specifically with CXF ?
It is a child's play though I guess I should not refer to such an advanced feature like that :-).
Whenever your custom OAuthDataProvider returns a new ServerAccessToken, a client-centric ClientAccessToken representation is returned to the client. At this point of time the data provider may choose to offer a refresh token too, which will be made available to the client.
On the server the refresh tokens are bound to a specific Client instance, I'm considering updating the Client model class to actually keep a list of refresh tokens, I'm not exactly sure yet if it will make it easier for the implementers or not, but in meantime one would need to keep a dedicated table joining client id plus the refresh and access token pairs.
So, the client will use the access token to access the end user's resources and at some point the access request will fail due to the access token expiring. All one needs to do now is to issue a Refresh Token grant request to OAuth2 Access Token Service, and then do a single retry using a newly obtained access token.
This is all to it.
Please check the Client-side support section for the new example showing this retry attempt pseudo code, as well as review on how well-known OAuth2 grants can be currently supported in CXF.
Enjoy!
Refresh tokens offer an advanced support for the OAuth2-protected applications to force the clients to re-authenticate regularly without forcing them to go via the re-authorization step involving the end user which may not always be practical.
They can also help with effectively getting the original access tokens revoked and replaced with the modified access tokens, example, with the new opaque scopes limiting or extending the current client permissions or even replacing the token type itself, example, replacing Bearer with MAC, all dynamically.
During the refresh process, the actual refresh token may also be 'renewed' with the new value by having the so-called key-rotation scheme implemented.
One possible approach is to have a short-lived access token and long-lived refresh token. Whenever the access token expires the client is forced to refresh with a valid refresh token and re-authenticate along the way.
I'd like to encourage developers to check OAuth2 archives on some interesting insights from the security experts on why using refresh tokens might benefit a particular OAuth2 application.
All this sounds good but you may want to ask, how to actually work with the refresh token in practice, specifically with CXF ?
It is a child's play though I guess I should not refer to such an advanced feature like that :-).
Whenever your custom OAuthDataProvider returns a new ServerAccessToken, a client-centric ClientAccessToken representation is returned to the client. At this point of time the data provider may choose to offer a refresh token too, which will be made available to the client.
On the server the refresh tokens are bound to a specific Client instance, I'm considering updating the Client model class to actually keep a list of refresh tokens, I'm not exactly sure yet if it will make it easier for the implementers or not, but in meantime one would need to keep a dedicated table joining client id plus the refresh and access token pairs.
So, the client will use the access token to access the end user's resources and at some point the access request will fail due to the access token expiring. All one needs to do now is to issue a Refresh Token grant request to OAuth2 Access Token Service, and then do a single retry using a newly obtained access token.
This is all to it.
Please check the Client-side support section for the new example showing this retry attempt pseudo code, as well as review on how well-known OAuth2 grants can be currently supported in CXF.
Enjoy!
Subscribe to:
Posts (Atom)
