[OT] You worked with CXF, Have a Great Time Out

Apache CXF has had a good year, with thanks to you using it, reporting the issues, asking the questions, with some of you spending a lot of your own time on helping improving it and keeping it flying and alive.

Time has come for a well deserved break. Have a Great Time Out, enjoy it !

Bean Validation for CXF JAX-RS Proxies

If you work with CXF JAX-RS Client proxies and have been thinking for a while, wouldn't it be good to have the proxy method parameters validated with the Bean Validation annotations, before the remote invocation is done, then a good news is, yes, starting from CXF 3.1.9 (which is due soon) it will be easy to do, have a look at this test please.

If you have the code which is considered to be safe with the respect to how it initializes the entities which will be posted to the remote targets then the client side bean validation might not be needed.

It can be more useful for the cases where the proxies are collecting the data from the databases or some other external sources - using the bean validation check to minimize the risk of posting some not well initialized data can indeed help.

I'd like to thank Johannes Fiala for encouraging us to have this feature implemented.

Fediz OIDC Story will continue at Apache Con EU 2016

ApacheCon Europe 2016 will be held in Seville Spain, Nov 16-18, with Apache Big Data starting on Monday Nov 14.

Colm and myself will continue talking about Fediz OpenId Connect following our presentation earlier this year.

Would you like to hear about our continuous effort to make the development of OpenId Connect applications with the help of Apache CXF OIDC, OAuth2 and JOSE code nearly as easy as writing a simple JAX-RS server and contributing to the idea of making OIDC going mainstream ?

Interested in making your own application server going OIDC way but concerned about the development costs ? See how Fediz IDP became OIDC-ready fast.

You are interested in the WEB security, and thinking about  where to start contributing to ?

Join us :-).  At the very least join all of us, listen to many interesting talks from my Talend, CXF and Apache SF colleagues. See you there !

CXF JAX-RS 2.0 - Perfect HTTP Spark Streaming Connector

Even the most conservative among us, the web services developers, will be better off admitting sooner rather than later that Big Data is not something that can be ignored, it has become a major technology in the software industry and will continue becoming even more 'influential' with the Internet of things wave coming in.

Where will it place your typical HTTP service which GETs some data for the users from some data store or accepts some POSTs with the new data ?

While I'm somewhat concerned seeing BigData consumers collecting the sources via a variety of custom optimized protocols and low-level transports like TCP, I firmly believe HTTP connectors should and will play a big role in connecting the WEB users with the Big Data processing chains.

HTTP, being so widely used, is a perfect frontend to the local networks where the nodes process the data, and while HTTP is synchronous for a typical interaction, JAX-RS 2.0 REST services can be quite smart. A variety of typical REST patterns can be employed, for example, a POST request handler with the data to be run through a BigData chain can let the application thread deal with it while respond to the user immediately, offering a link with a job id where the status can be monitored or the results returned from.  Or the handler can rely on the suspended HTTP invocations and start streaming the results back as soon they become available.

I have created a Spark Streaming demo showing some of the possible approaches. This demo is a work in progress and I will appreciate a feedback from the Spark experts on how the demo can be improved.

The demo relies completely on JAX-RS 2.0 AsyncResponse - the typical pattern is to resume it when some response data are available - and what is good it can be suspended multiple times to do a fine grained optimization of the way the service code returns the data. StreamingOutput is another piece - it allows writing the data back to the user as soon as they become available.  FYI, CXF ships a typed analog, called StreamingResponse, you can see how it is being indirectly used in this RxJava Observable test code.

But let me get back to the demo. It shows two types of Receivers in action.  This demo service shows how an HTTP InputStream can be converted to a List of Strings with a custom Receiver making them available to Spark. The service currently creates a streaming context per every request which I imagine may not be quite perfect but my tests showed the service performing quite well when the input set is parallelized - less than a sec for a half of MB PDF file.

Speaking of PDFs and other binary files. One of the service methods uses a beautiful Apache Tika Parser API which is wrapped in this CXF extension. These few lines of code is what it takes to have a service enabled for it to push the content of either PDF or OpenOffice documents to the Spark pipeline (I only added PDF and OpenOffice Tika Parser dependencies to the demo so far). I'm sure you are now starting wondering why Tika API is still not used in your JAX-RS services which parse PDF only with the PDF specific API :-)

I keep getting distracted. Back to the demo again. This demo service is a bit more closer to the real deployment scenario. It uses a default Spark Socket receiver - JAX-RS 2.0 service, being a good HTTP frontend, forwards the HTTP stream data to the internal Spark Streaming TCP server which processes the data and makes them available to a JAX-RS AsyncResponse handler which is also acting as a Socket server. The correlation between a given HTTP request and the Spark output data is achieved with a custom protocol extension. I can imagine it will be easier with an internal Kafka receiver which is something that the demo will be enhanced with to try later on.

In both cases, the demo streams the response data pieces back to the user as soon as they become available to the JAX-RS AsyncResponse handler.

Additionally, the demo shows a CXF JAX-RS Oneway extension in action. HTTP Client will get a 202 status back immediately while the service will continue with processing the request data.

I'm sure the demo will need more work but I also hope there's enough material there for you to start experimenting. Give it a try please and watch for the updates. I think it will be very interesting to see how this demo can also be written with Apache Beam API, check this blog entry for the good introduction.

Enjoy ! 

Progress In The JAX-RS 2.1 space

For those of you thinking what is going to happen to JAX-RS a good news is that JAX-RS 2.1 will live, surely it was the only possible outcome given the quality and the popularity of JAX-RS 2.0.

Check out this Java One 2016 Key Note, and I will continue standing by my assertion that even more is to come from JAX-RS.  


As far as Apache CXF is concerned, Andriy  has been working hard on a JAX-RS 2.1 branch where he implemented a 2.1 Server Sent Events API. And after Marek released the very first JAX-RS 2.1 API artifact to Central Andriy merged his work to the CXF 3.2.0 master branch.

It is very good because we can now start working toward releasing CXF 3.2.0 with this early JAX-RS 2.1 API to be implemented for the CXF users to experiment with - JAX-RS 2.0 users will be able to migrate to CXF 3.2.0 without changing anything in their services code.

The new JAX-RS 2.1 features (SSE, NIO and Reactive Invokers, with the early API improvements likely to happen during the coming specification work) are cool and it is worth taking the hats off to the engineering minds of the Jersey team for the top work they did.
  

[OT] Become Most Enigmatic Person in The Office - Discover CXF

Going the winding Apache CXF path is not that scary for a web services developer - some features may not be there just yet but you may discover something new instead, while helping driving the CXF forward along the way.

Have no fear, answer the call, help your team discover what Apache CXF really is. And become the most popular and enigmatic person in your office :-)

SwaggerUI in CXF or what Child's Play really means

We've had an extensive demonstration of how to enable Swagger UI for CXF endpoints returning Swagger documents for a while but the only 'problem' was that our demos only showed how to unpack a SwaggerUI module into a local folder with the help of a Maven plugin and make these unpacked resources available to browsers.
It was not immediately obvious to the users how to activate SwaggerUI and with the news coming from a SpringBoot land that apparently it is really easy over there to do it it was time to look at making it easier for CXF users.
So Aki, Andriy and myself talked and this is what CXF 3.1.7 users have to do:

1. Have Swagger2Feature activated to get Swagger JSON returned
2. Add a swagger-ui dependency  to the runtime classpath.
3. Access Swagger UI

For example, run a description_swagger2 demo. After starting a server go to the CXF Services page and you will see:

Click on the link and see a familiar Swagger UI page showing your endpoint's API.

Have you wondered what do some developers mean when they say it is a child's play to try whatever they have done ? You'll find it hard to find a better example of it after trying Swagger UI with CXF 3.1.7 :-)

Note in CXF 3.1.8-SNAPSHOT we have already fixed it to work for Blueprint endpoints in OSGI (with the help from Łukasz Dywicki).  SwaggerUI auto-linking code has also been improved to support some older browsers better.

Besides, CXF 3.1.8 will also offer a proper support for Swagger correctly representing multiple JAX-RS endpoints based on the fix contributed by Andriy and available in Swagger 1.5.10 or when API interface and implementations are available in separate (OSGI) bundles (Łukasz figured out how to make it work).

Before I finish let me return to the description_swagger2 demo. Add a cxf-rt-rs-service-description dependency to pom.xml. Start the server and check the services page:

Of course some users do and will continue working with XML-based services and WADL is the best language available around to describe such services. If you click on a WADL link you will see an XML document returned. WADLGenerator can be configured with an XSLT template reference and if you have a good template you can get UI as good as this Apache Syncope document.

Whatever your data representation preferences are, CXF will get you supported.

 

CXF Spring Boot Starters Unveiled

The very first check some new users may do these days, while evaluating your JAX-RS implementation, can be: how well is it integrated into SpringBoot ?

And the good news is that Apache CXF 3.1.7 users can start working with SpringBoot real fast.
 
We have left it somewhat late. It is hard to prioritize sometimes on various new requirements. And see some users moving away. In such cases the community support is paramount. And the Power of Open Source Collaboration came to the rescue once again when it was really needed.

I'd like to start with thanking James for providing an initial set of links to various SpringBoot documentation pages and reacting positively to the initial code we had. But you know yourself - sometimes we all value some little 'starters' - the initial code contributions :-)

And then we had a Spring Boot expert coming in and getting the process moving. Vedran Pavic helped me to create the auto-configuration and starter modules for JAX-RS and JAX-WS, patiently explained how his initial contribution works, how these modules have to be designed, and helped with the advice throughout the process. I felt like I passed some SpringBoot qualification exam once we were finished which let me continue enhancing the JAX-RS starter independently before CXF 3.1.7 was released.

CXF Spring Boot starters are now documented at this page which is also linked to from a Spring Boot README listing the community contributions.

If you are working with CXF JAX-RS then do check this section. See the demos and get excited about the ease with which you can enable JAX-RS endpoints, their Swagger API docs (and auto-link Swagger UI - the topic of the next post).

See how you can run your CXF WebClient or Proxy clients in Spring Boot, initialized if needed from the metadata found a Netflix Eureka. The demo code on the master uses a CXF CircuitBreakerFailoverFeature written by a legendary DevMind, a sound, simple and light-weight Apache Zest based implementation.
Not all users may realize how flexible CXF Failover Feature is. 

While the most effort went into a JAX-RS starter I'm sure we will add more support for JAX-WS users too.

We'll need to do a bit more work - link CXF statistics to the actuator endpoints, support scanning JAX-RS Applications and few other things.

If you prefer working with Spring Boot: be certain that a second to none support for running CXF services in Spring Boot will be there. Enjoy!

[OT] Reuse Or Reimplement ?

I said in one of my earlier posts I'd share some thoughts I've had during the years on re-using vs re-implementing while working on various CXF projects. Some of it may be a bit CXF specific but most of it might be interest to non-CXF developers too.


When the time comes to implement a new feature the immediate decision that needs to be taken is how you do it. In general it is always a good idea to re-use a quality 3rd party library that can help in realizing the new feature fast.


Consider a task of providing a UI interface to have Swagger JSON documents nicely presented. One can invest the time and write UI scripts and pages. Or one can download a well-known Swagger UI module.

Another example: one needs a collection sort algorithm implementation which will do faster than Java Collections code. One can invest a time and write a new library or look around and try an Apache or Google library.

In both cases re-using the existing solution will most likely be better and help deliver the higher-level, complete offering faster.

Things may get more complicated when one works on a project in a competitive space. For example, at some point there were 6 active JAX-RS Java implementation projects, with other non JAX-RS implementations such as the one offered by Spring adding up to the total number.

When you work on a project like that one a number of important decisions need to be made: how complete you'd like your project to be ? Is supporting HTTP verbs and reading and writing the data is all what is needed ? What sort of security support around the service you'd like to provide ? What other extensions should your project have ? How would you like your project be recognized - as a library or something bigger that offers all sort of relevant support for people writing the HTTP services ?

The higher the 'ambitions' of such a project the more likely the 're-implementing' becomes a viable option, nearly a necessity in some cases. In fact re-implementing is going all around at such projects.

I've been involved in a fair bit of re-implementation projects.

To start with we started implementing JAX-RS at a time when Jersey was already high. Why ? To have Apache CXF open to users with different preferences on how to do HTTP services. It was hard at times but it was really never simply because we wanted to prove we could do it.

The latest 're-implementation' was JOSE. Why ? I won't deny I was keen to work with the low-level security code closer, but overall, I wanted a CXF Security Story be more complete. Implementing it vs re-using the quality libraries I listed at the Wiki let us tune and re-work the implementation for it to be better integrated with the JAX-RS and Core security support so many times that it would be highly unlikely to happen if I were working with a 3rd party library.

I do not think re-implementing in an open way is not healthy. For example it has been acknowledged that having many JAX-RS implementations around helped to make JAX-RS more popular. Re-implementing may offer more options to users.

Or, reimplementing can prove a complete loss of time. Here are some basic 'guidelines' if you decide to try to re-implement in the Open Source:
- think not twice but many times before you try it
- if you feel the urge then do it, get the experience, make the mistakes, next time you will do the best choice
- never expect that once you re-implement something then everyone will stop using whatever they use and switch to what you have written - a lot of clever developers are working full time
- if you'd like others to use your project then you absolutely must love working with the users, don't even start if you think that it will be up to the Customer Support
- you need to have a support of your colleagues
- expect that the only 'remuneration' you will have is the non-stop work to keep the project constantly evolving

Yes, very often re-using may be the very best thing :-)

Enjoy, Happy Re-Using, Happy Re-Implementing :-)



 

Asynchronous JAX-RS Proxies in CXF

Dan had an idea the other day to get CXF JAX-RS proxies enhanced a bit for them to support the asynchronous calls. After all, HTTP centric JAX-RS 2.0 and CXF WebClient clients support such calls with AsyncInvoker.

So here is what we have started from. Simply register InvocationCallback with a proxy request context as shown in the examples and make the asynchronous call. The proxy method will return immediately and the callback will be notified in due time once the typed response is available. As the examples show one can register a single callback or a collection of callbacks bound to specific response types.

I suppose we can consider generating typed asynchronous proxy methods from the service descriptions such as WADL going forward.

This feature will be available in CXF 3.1.7. Give a try please, refresh your JAX-RS proxy code a bit, enjoy. 

Apache CXF JAX-RS and SAML Assertions

While the software industry with the interests in WEB security is enthusiastically embracing the latest and coolest technologies such as OpenId Connect and JOSE, with JSON Web Tokens being the stars of the advanced security flows, less 'glamorous' SAML security tokens have been continuing helping to secure the existing services.

CXF JAX-RS has been providing a comprehensive support for SAML assertions for a while now which is being relied upon in a number of productions. I'd also like to encourage the developers who work with SAML give this access control feature a try.

The question which is often being asked is how a JAX-RS client gets these assertions. Please read this informative blog post explaining how CXF JAX-RS clients can seamlessly get a SAML assertion from a WS STS service and use it with the server validating it against STS or locally.

Please also check this section if you are you curious how to reuse SAML assertions in OAuth2 flows.

  

Practical Cryptography with Apache CXF JOSE

It has been a year since I had a chance to talk about Practical JOSE in Apache CXF at Apache Con NA 2015.

We have significantly improved CXF JOSE implementation  since then, with Colm helping a lot with the code, tests, documentation. The code has become more thoroughly tested, the configuration - better, with the documentation being updated recently. 

Production quality CXF STS service can now issue JOSE-protected JWT assertions and Fediz OpenId Connect project directly depends on JOSE in order to secure OIDC IdTokens.

But it is important to realize that doing JOSE does not mean you need to do OAuth2 in general or OpenId Connect in particular, though it is definitely true that understanding JOSE will help when you decide to work with OAuth2/OIDC.
As such, a web service developer can experiment with JOSE in a number of ways.

One approach is to use JWS Signature or JWE Encryption helpers to sign and/or encrypt the arbitrary data.

For example, have your service receiving a confidential String over 2-way HTTPS, then JWE-encrypt and save it to the database to ensure the data is safe or JWS-sign only and forward further, being assured the data won't be modified, and choose between JWS Compact or JSON representations.

Have you already heard JOSE sequences have the data Base64 URL encoded ? Try JWS JSON with an unencoded payload option.

Another approach is let CXF do JOSE for you. Use CXF JOSE filters and make service data secured by typing few lines of text in the configuration properties.
These filters will do the best effort at streaming the outbound data while preparing JOSE sequences.

Would you like to link client JWT assertions obtained with the progressive services such as CXF STS to the data being protected ? Add a couple of filters. 

I honestly think that JOSE is the best technology which can help many of us  understand better what cryptography is.

Start with selecting a signature algorithm. You most likely have a Java JKS key store somewhere around, so go for 'RS256'. Get the private key out and sign, then get a public key and validate as shown here.
Next try to encrypt, select RSA-OEPA to make it real fast given that you have this JKS store. Use a public key to secure a content encryption key generated by CXF for you and then do A128GCM content encryption. Finish with decrypting the content with a private key.

Works ? Interested in trying different key sizes or combinations of JOSE algorithms ? No problems, try them fast. Learn more about these algorithms next. See how it all works when the CXF JOSE filters do the work.

We've thought a lot on how to help developers start experimenting with JOSE as fast and easy as possible and I hope those of you who will start working with CXF JOSE code will help us make it even better.

Would like to use some other quality JOSE libraries such as these ones ?  No problems, use them inside your custom JAX-RS filters or directly in the service code.

You may say, I'm not really seeing others use JOSE in regular HTTP services work. Let me finish with this advice: please do not worry about it, be a pioneer, experiment and find new interesting ways to secure your services and prepare them to work in the world of JOSE-protected tokens and data flowing everywhere.

Do JOSE today, convince your boss your team needs it :-), become a cryptography expert. Enjoy !

Observations about Apache Con NA 2016

This year Apache Con NA was held in Vancouver BC.

As usual, being at Apache Con gives a chance to talk to your fellow Open Source developers and this year it was as great as ever - meeting my old and new Talend colleagues, talking to those I already have met before and getting to know other people attending the conference was nice. 
The conference hotel was few hundred meters away from the waterfront where one could walk or run to a green Stanley Park:

Now let me talk about the actual conference. Big Data and Core conference are no longer run at the same time, with a single day intersection only. I guess I was preferring a more compact 'mix-in' format as I could attend to either BigData or Core presentations in a fewer number of days. But organizing a successful conference is very difficult - in the end of the day whatever format works best for Apache Con is the winning format.

I did like and learn something new from all the keynotes I listened to but I particularly enjoyed an Open Source is a Positive Sum Game by Sam Ramji. Have you ever  looked at the schedule, not sure what to expect from the listed talks, and then someone starts speaking and you realize you are listening to a visionary ?  This is what I felt when listening to Sam.

A number of other talks were interesting. My colleague JB's presentations were both interesting and entertaining, and I was also happy to see Hadrian and Jamie, both my former colleagues :-), co-presenting. 

I think Colm and myself had a good audience during our presentation. It must've been difficult for those who attended to listen to a lot of security related information presented on Friday after lunch :-) and we are grateful to all who were there. I did overrun by 1 minute though and we had no chance to talk to the audience afterwards but we did convey a lot of information during our talk.

And then finally we had the last few presentations to choose from and we made it to a Shawn McKinney's presentation. Now imagine it is 16.00, late Friday afternoon, and you are about to listen to yet another security related talk :-). I think Shawn did remarkably well. Shawn's down to earth, likeable presentation style made the real difference. And while I did learn few things about Role Based Access Control (such as the temporal restrictions), what really did get to me were Shawn's advices to "test and re-use".  You may say it is all quite obvious but sometimes one can get lax on either of those fundamentals, myself including. I'd like to talk about some of the thoughts I've had about the 're-use vs implement yourself' later on.

It was great to be there :-)




   

Talking about Fediz OIDC at Apache Con NA 2016

Colm and myself are going to talk about Fediz OpenId Connect at Apache Con NA 2016. The session is on Friday 13th May.

Be there if you can, you can then tell your grandchildren you were at the 1st public presentation about Fediz OIDC :-)

I do look forward to being at Apache Con again. Seeing and talking to the colleagues from Apache CXF and other projects is always super great.

[OT] U2 Innocence And Experience or Understand HTTP services with CXF

I've already told to all of my colleagues who would listen how lucky I was to get a chance to listen live to U2 who played several concerts in Dublin as part of their Innocence and Experience tour. 

I've already told why I like U2. But seeing them playing live is really special. The voice is so good it is shocking at first. They are hard working and innovative, despite not being that young any more, the latter part is something I can definitely associate with :-).

In all of the [OT] entries on my blog I'm trying to look for a 'connection' to Apache CXF. No exception this time:

Apache CXF is not only a place where one can have a Web/HTTP Service created. But also go from a Novice to Expert in building such services. CXF may not offer a way for a Hello World application be created for you without doing anything at all. But it has been known to deliver in supporting most demanding services. By the time the developers have those services up and running they have become the experts who know what it takes to write a service that works well. They have moved from the 'Innocence' of Hello World services to 'Experience' required to support Real World services. 

 

 

CXF Master JAX-RS 2.1 Branch is Opened

Good news for CXF JAX-RS users: Andriy Redko has opened a CXF Master JAX-RS 2.1 branch. Server Side Events is the first feature of 2.1 API which is supported on this branch. Having this 2.1 API Snapshot is handy.

The development of JAX-RS 2.1 has been frustratingly slow but there's some progress nonetheless with Jersey (RI) expected to be ready as soon as realistically possible, given that all the major features proposed for JAX-RS 2.1 have already been implemented in Jersey.

JAX-RS is easily the best API for building REST clients and servers. Despite the process difficulties it will continue evolving. Use it and believe more is to come in the JAX-RS space.

New Kid On The Block: Fediz OpenId Connect

Apache Fediz, Identity Provider for the WEB, was created by Oliver Wulff and during the last few years, with the major support from Colm and Jan, has become quite a popular provider for supporting SSO with the help of the WS-Federation Profile.

Before I continue, I'd like to clarify that even though WS-Federation is obviously related to SOAP, the important thing is that as far as the user experience is concerned, it is pure SSO. For example, AFAIK, a Microsoft Office Outlook login process is currently WS-Fed aware.

But OpenId Connect (OIDC) is a new SSO star for the WEB, with all of the software industry players with SSO-related interests supporting it, as far as I can see it.

OIDC really shines. I was talking about something similar before in context of the JOSE work, it is really been designed by some of the best security and web experts in the industry. And OIDC is still a very bleeding edge development as far as a maintsream adoption by the software industry is concerned. Google, Microsoft, and other top companies have created OIDC servers, but what if you want your own OIDC ?

Fediz OpenId Connect (Fediz OIDC) is the new project that Colm, Jan and myself started working upon back in November 2015 and it joins a family of OIDC-focused projects that are appearing probably every month in various developer communities.   

As you can imagine we are at the start of a rather long road. OIDC is great but is undoubtedly complex to implement right.  We've had a good progress so far and most of OIDC Core is supported OOB, something that you can try right now.

Apache CXF OAuth2 and OIDC authorization modules are linked to a flexible Fediz IDP (Authentication System) with the minimum amount of code. We will be working on making it all more feature complete, robust, configurable, customizable, production ready.

We are planning to talk about Fediz OIDC a lot more going forward.

Stay tuned !