Thursday, May 10, 2018

Thorntail Bird Brings New Day in Container Development

WildFly Swarm team started thinking about renaming the project awhile back, with the first message to the community coming in February.

Ken and Bob have put a lot of effort into finalizing the process, with the announcements starting coming this week, the week of Red Hat Summit.

The most exciting part is the fact that a new Thorntail Project GitHub organization has been created with the main concepts of this new 4.x line being captured here.

It is going to be a very cool project. For a start, many tests run in Docker Machine. The new project already ships and will add the best and proven components, ready to run in the micro services and Cloud centric environments. It is expected to be light-weight and easy to work with. I hope you are already forking the repository while you are reading this post :-).

Here Comes Another New Day. New Day in the application container development. Start enjoying it now !

Tuesday, February 13, 2018

Adieu Talend, Hello Red Hat !

Seems it was nearly yesterday when I joined Talend seven years ago. Time has flown so fast... Next week I will be returning to Red Hat but first I will talk a bit about my years with Talend.

I'd like to believe that working for Talend has helped me become a better engineer, grow in confidence.  And what about those unforgettable Talend R&D events :-) ? No doubt, it has been an interesting and exciting journey.

It has not been easy to find a link to a piece of music which would associate well with the company, but I think I've got it in the end. The text there is a bit sombre, but the music reflects well what I'd like to remember about Talend, the energy and the style: enjoy Ave Cesaria By Stromae. Thank you Talend, Goodbye.

And now I'll be heading back to Red Hat :-). I will be joining a WildFly Swarm team and I look forward to and optimistic about it and the new challenge. I'll have to learn new things. I will enjoy it too. And in time, after I settle well, I will return to this blog and talk about WildFly Swarm and other related projects.

Stay Tuned !

Thursday, February 8, 2018

Apache CXF Story Will Continue

When I started working on Apache CXF full time it was already a well established project, shipping a production quality JAX-WS and an early JAX-RS implementations.

During the next N years, with some short breaks, all of us did put a lot of effort into supporting the CXF community,  keeping enhancing the JAX-RS, various security features, fixing lots and lots of bugs, and trying to support the idea that "CXF was more than just a library" :-).

I'm curious, how many SOAP or pure HTTP calls have been made over the years with the help of CXF ? Sometimes one can read: "This product supports thousands of transactions per minute". Would be fun to read somewhere that "CXF has supported several millions of service calls over 10 years" :-). Or how many downloads have been made ? Who knows...

It is satisfying to see that today the users keep coming to CXF and ask questions and open new issues. No doubt it has helped many users, helped to completely mainstream JAX-WS and then JAX-RS, alongside its Jersey and RestEasy 'colleague' frameworks.

No doubt the Apache CXF story will continue and I've been happy to be part of this story. Thank You !

Wednesday, December 27, 2017

[OT] Work Next Year the way Arsenal and Liverpool played their football

I've been lacking the ideas on how to write the regular and last off-topic post of the year, thinking which piece of music I should link to.

And then the inspiration came from the completely unexpected source.

Those of you who follow Arsenal FC in the English Premier League know that Arsenal can either draw against Liverpool but in such a way that fans will remember it for years (yes, that Liverpool 4 to Arsenal 4 draw), or, most likely, lose badly to this Merseyside team, something like 1:4 or similar.

So less than a week earlier, Arsenal was playing with Liverpool in London, losing 0:2. Oh well, most Arsenal fans thought, one of those days which can only be described in Fever Pitch. Then, in the 2nd half, after the few minutes, while the fans were having mince pies and tea,  Arsenal were 3:2 up, with Liverpool managing to equalize. The game saw many mistakes and brilliant moves and fans just had the day of the year watching the game. I liked this summary.

How would I translate that to a New Year wish for you, the software engineers ? Here it is:

Enjoy your work next year, try to do something extraordinary, something new, and don't be afraid to make mistakes :-)

Happy New Year !


Get OpenAPI v3 JSON with CXF Now !

The Apache CXF has done some initial work to have OpenAPI v3 JSON reported from the JAX-RS endpoints.

Andriy Redko has started with the OpenApiFeature implementation which depends on the latest OpenApi v3 aware swagger-core and swagger-jaxrs libraries and demoed it here.

In meantime I did a Swagger2 to OpenApi v3 JSON conversion filter which reacts to openapi.json queries by converting Swagger2 swagger.json produced by Swagger2Feature to openapi.json with the idea of making it easier for the existing upstream code (which has already integrated Swagger2Feature) to start experimenting with OpenAPI v3, before doing the switch to the new feature (and dependencies).  

This effort is still a work in progress but the results in both cases are promising. The new feature and the conversion filter will require some more improvements but you can start experimenting with them right now. And if you are someone like me then you will be positively surprised that SwaggerUI  3.6.1 and newer can handle both Swagger2 and OpenAPI v3 JSON :-).

Enjoy !

Simple Reuse of org.reactivestreams in CXF

I mentioned earlier that one could link RxJava2 Flowable with JAX-RS AsyncResponse with Subscriber which will do the best effort at streaming the data pieces converted to JSON array elements, see this example.

That works but requires the application code refer to both JAX-RS AsyncResponse and CXF specific JsonStreamingAsyncSubscriber (RxJava2 specific at the earlier stage), as opposed to simply returning Flowable from the resource method.

In meantime, John Ament added the initial Reactor integration code, and as part of this work John also provided the org.reactivestreams compatible JsonStreamingAsyncSubscriber to be optionally used with the CXF Reactor invoker.

As a result we've found the opportunity to do some refactoring and introduce the simple org.reactivestreams utility module which is now reused between CXF RxJava2 invoker and Reactor invoker: the common invoker code both invokers delegate to will check if JSON is expected and if yes then will register JsonStreamingAsyncSubscriber as org.reactivestreams.Subscriber with org.reactivestreams.Publisher which can be either RxJava2 Flowable or Reactor Flux.

The end result is that users can now write simpler code by returning Flowable or Flux from the service methods.

It is an interesting but simple example of reusing the org.reactivestreams aware code between different org.reactivestreams implementations.

Tuesday, September 12, 2017

The Real Data Processing with Apache Beam and Tika

If we talk about the data injestion in the big data streaming pipelines it is fair to say that in the vast majority of cases it is the files in the CSV and other text, easy to parse formats which provide the source data.

Things will become more complex when the task is to read and parse the files in the format such as PDF. One would need to create a reader/receiver capable of parsing the PDF files and feeding the content fragments (the regular text, the text found in the embedded attachments and the file metadata) into the processing pipelines. That was tricky to do right but you did it just fine.

The next morning you get a call from your team lead letting you know the customer actually needs the content injested not only from the PDF files but also from the files in a format you've never heard of before. You spend the rest of the week looking for a library which can parse such files and when you finish writing the code involving that library's not well documented API all you think of is that the weekends have arrived just in time.

On Monday your new task is to ensure that the pipelines have to be initialized from the same network folder where the files in PDF and other format will be dropped. You end up writing a frontend reader code which reads the file, checks the extension, and then chooses a more specific reader.   

Next day, when you are told that Microsoft Excel and Word documents which may or may not be zipped will have to be parsed as well, you report back asking for the holidays...

I'm sure you already know I've been preparing you for a couple of good news.

The first one is a well known fact that Apache Tika allows to write a generic code which can collect the data from the massive number of text, binary, image and video formats. One has to prepare or update the dependencies and configuration and have the same code serving the data from the variety of the data formats.

The other and main news is that Apache Beam 2.2.0-SNAPSHOT now ships a new TikaIO module (thanks to my colleague JB for reviewing and merging the PR). With Apache Beam capable of running the pipelines on top of Spark, Flink and other runners and Apache Tika taking care of various file formats, you get the most flexible data streaming system.

Do give it a try, help to improve TikaIO with new PRs, and if you are really serious about supporting a variety of the data formats in the pipelines, start planning on integrating it into your products :-)

Enjoy!