Solid Craft

GeeCON 2012 review

noreply@blogger.com (Jakub Nabrdalik) — Mon, 21 May 2012 14:26:00 +0000

Bruce Almighty!

GeeCON 2012 is over. This year I've decided to go for 3 days, where the first one was a „University day”, and the other two were regular conference days. „University day” is a misleading name, you'd think it's something for students, for beginners perhaps. The idea, as far as I know, was to have a day full of workshops, but as there was just one bring-your-own-laptop TDD workshop, it turned out to be a „normal” conference day, with 2h talks.

Now, „normal” doesn't mean bad. Quite the opposite.

As usually, this is my little review of the conference. Take into account, that with up to 5 parallel tracks, I've seen no more than 25% of the whole event.

Wojtek Erbetowski, pointed out, that as we gain knowledge, it's getting harder and harder to find something amazing at a conference. I was overwhelmed with new knowledge, the first time I went to one, but since I try to keep up to date, since I visit up to six conferences every year, and since speakers tend to give the same talks all over the country, I consider a conference fruitful if I take one thing that can help me in a substantial way.

I'm happy to report, that it was the case with GeeCON.

University Day started for me with Dierk König talking about concurrent programming in Groovy (GPars library). Dierk is a commiter to GPars and a co-author of „Groovy in Action”. Chances are, that if you're using Groovy, you already know the lib, but if you don't, it's got pretty a lot of ways to deal with parallel programming without touching threading manually. It's got STM (Software Transactional Memory), Actors, Agents, Communicating Sequential Processes (CSP), dataflow stuff and more. There's even a thing called KanbanFlow. Yeah, the Kanban in name is not accidental. And the best part: all of this is relativly easy to understand and use.

His talk was good, but to tell the truth, I've seen a GPars presentation at GeeCON 2011, by Václav Pech („Pick low-hanging fruit"), and even though Václav had just an hour, I think his talk was better.

Unfortunatelly, the following Dierk's talk: „Pro Groovy”, was both short (just 1h), and shallow. Dierk was talking about Abstract Syntax Tree transformations, but while I hoped for a show of how to write AST Transformations, Dierk managed only to show a few build in annotations, that were handled by them.

I wish Dierk was available for a whole day. If you are a planning a conference and thinking about whom to invite, please consider asking Dierk and giving him a whole day for a really pro workshop. That would make my day.

Gavin King and Stéphane Épardaud were introducing us to Ceylon, a new programming language on the JVM. If you are wondering, why should you care about this one, you shouldn't. Give it a few years, and check again. The talk was nice, though.

Then came Bruce Eckel with „Atomic Scala”. Bruce is writing a new book, and the idea he has is to create something for complete newbies in programming, allowing them to learn Scala as the first programming language. The goal is noble, but that means you shouldn't probably buy it, because it's way faster to explain a lot of language constructs comparing to another language you know, than to explain it from the ground up. Bruce's talk was too basic even for such a dumbass as me.

Bruce also gave the opening talk “The Power of Hybridization”, the next day, and thought it lacked a thesis (Bruce had to finish at about 70%), his deep, multilingual perspective and tons of anecdotes, made it very interesting to me. By the way, if you've read about how slides should look like (no text-reading, only slogans, full screen pictures, etc.) Bruce is doing exactly the opposite thing. Nobody cares, when you have something to tell, though.

I'll skip Ivar Jacobson's “Liberating the Essence...” talk, as it lacked.. the essence. I'll also skip Gavin King's “Ceylon: the thinking behind a new language”, as it was just the same thing as with any other Java-replacement language I know of. Let's move to Adam Bien talking Web Containers and Java EE.

Terminator @pub in Poznań

Adam Bien is a Java Rock Star, Champion, expert, writer, consultant and what else, so you'd correctly assume, his perspective will be very orthodox, especially when you consider that there were at least 4 other languages represented at the conference (Scala, Groovy, Ceylon, Kotlin). But he made a point – the main reason we take all those libs/languages, we change EE containers to web containers + frameworks, is that we do not know Java EE 6 well enough (before 6, using pure Java EE was masochism at best). And then we tend to reimplement/rediscover the things Java EE has already solved.

The thing he misses, though, is that it's easier to learn and use those “other” technologies, than do proper Java EE. And yes, after years of corpo-failures, programmers tend to be biased against Java EE. Thus, while Java EE is finally good, and it makes perfect sense to stay with Java EE, if you are a Java EE master-blaster, I doubt you'll be as productive as with for example Grails. You'll do good to pick whatever you'd like to learn, and learn it throughly.

Adam's talk was refreshing. Apart from guys from SoftwareMill, I haven't seen anyone talking about Java EE 6 with such expertise and sanity.

And then came my favorite talk. Sam Newman from Thoughtworks, talked about design and architecture with emphasis on rapid release.

Wojtek Erbetowski showing a prototype
of an operating system on top
of a prototype of a hardware

There aren't many talks about architecture at Java conferences. In fact, there are hardly any. I guess it's easier to fill one hour showing some language basics or a library, than to explain and explore an architecture and its consequences. Thoughtworks was present at 33rd, with James Lewis and micro services, and I see that the lessons they've learned, are being propagated to other projects. I won't go into details of micro services, as I've already done that reviewing James' talk. Sam was happy to present a case of how to use system decomposition (micro-services at a service per Bounded Context level), and what the effects are.

Some of his learned lessons were:

beware of shared lib and serialization protocols (evil for microservices)
do not send internal representation of data across the wire (things change)
fuck WSDL, go with simple XML, Xpath, and let every service prepare stub-tests, so you can safely change the schema when you want to
codify the contract between services with writing a test (the reason is the same as above)
do not share database between services; you'll make it impossible to evolve them independently; you can use one database with different schemas
data should be owned by the service it uses (similar to above)

When I say I'm interested, and that I see potential here, I really mean it. I'm about to lead a development of a bank system, implementing such an approach. We are already using many small, dedicated to single purpose apps, written in different technologies (Java + Spring MVC, Scala + Play), that communicate via RESTful HTTP, and we are going to expand on that concept. We'll see whether we can keep it so simple as Thoughtworks implies. I'll let you know when we have some outcomes.

I was hopping between presentations for the next hour, as I have either already seen then somwhere else, or was uninspired after a few minutes. Then I went for James Williams, talking about Ratpack. I really feel sorry for James, he made quite a trip here, but as this was the last presentation this day, and he had guys like Bruce Eckel as an alternative, not many people showed up.

I've started the last of the conference with Thomas Sundberg and his rather basic talk about all the wrong things we can do to fail a project. As there was nothing new to me, I left early. Then I moved to Kevlin Henney to listen about OOP in Java (deja-vu? not really), and to Tomasz Kowalczewski talk about Closures in Java 8. That was similar to Simon Ritter at 33rd, but focused solely on Closures. Great talk, nice stuff, Tomasz did great work, though it's a bit disappointing that functions are not going to be first class citizens. Yet.

Oh, well, no reason to go back from Groovy, I guess.

And then there was Piotr Gabryanczyk with Scala, Akka and Camel. That was also a great talk, showing simplicity of programming using those technologies. I wonder how it fits Thoughtworks' micro-services, or are we getting too close to ESB and single point of bullshit that way.

Kevlin Henney closed the conference with examples of cool code. His talk was fantastic, funny, and inspiring. Especially examples. One of the best closing talks I've seen. Thanks.

And there were parties. But that's a completely different story.

Saturday was for Open Spaces, lead by Bruce Eckel, but I couldn't attend.

PS: Some pictures are courtesy of Michał Lewandowski. I've totally forgotten my camera.

33rd Degree day 3 review

noreply@blogger.com (Jakub Nabrdalik) — Sun, 25 Mar 2012 17:09:00 +0000

At the last day of the conference, I've decided to skip the first presentations, and get some sleep instead. I was afraid that Venkat's show is going to be too basic, I will see Jacek Laskowski talking about closure at 4Developers, which I'm kind of supervising from the Java perspective. As an afterthought, I should have gone to see "Static Code Analysis and AST Transformations" by Hamlet D'Arcy, as I really like this guy and the topic is intriguing at least, but I didn't want to intoxicate myself more with coffee, and really needed to get some sleep.

MongoDB: Scaling Web Application

The first talk I've seen was by Ken Sipe. It was a very interesting case study, and I love case studies. They give you a real perspective, unlike the product-half-marketing that some authors are doing, and unlike self-marketing that some "professional" speakers perform.

Ken had an interesting project at hand: a Facebook like social service, that deals with politic, named GoVote. Half of the presentation was a bit basic, with simple CRUD syntax, but when Ken started to talk about modeling, it got interesting.

Ken used Grails as the web framework and went through several plugins/libs to get him to the performance and efficiency he needed. Every one of them failed, except for GSP, which is a thing worth reflecting about. His lesson was not really surprising: you should not use any abstracts (ORM?), when playing with document databases. He sticked to GORM because of how fast his problems were solved on the mailing list and bug tracker, but GORM doesn't really give you the resolution you want, when dealing with tree structures composed of documents.

My favourite part, albeit short, was about modeling. Ken managed to get ONE hit to DB per dynamic web page, something I've always dreamed of. His advice was to focus on use cases, and design document model with use cases in mind. You get a sign of bad design, when you find nested joins (hitting more than once for the data), and you can usually get away with tags, instead. Interesting stuff, I hope to verify that soon.

HTML5 For Developers

Nathaniel Schutta had a large presentation at hand, nothing he kind fit into only one hour, so he gave us a choice: what do we want to hear about. It turned out, the audience was most interested into Web Workers, local web storage and Canvas. Each had their caveats.

Web workers allow to get some much needed concurrency into heavy GUI (or game engines, I guess), work with a simple API where you can handle errors and spawn new workers.

Local storage is unfortunately limited by default to 5MB, and you may get unsuccessful persuading the user to change it. Apart from that, it's a simple key-value store (both Strings), with storage events (think on-insert etc.), but current implementations handle those not without some problems.

Canvas was quite interesting. Though it has a very simple API, and only basic primitives implemented, I've already bought a 500 pages long book about it (and that's without any info on all the libs build on top of it).

Ending keynotes

There were three ending keynotes, one from Nathaniel Schutta, Jurgen Appelo and the last from Robert C. Martin. As expected, those were entertaining, though unless you've been sleeping through the last three years in a cave, you would not be surprised by both craftsman. Jurgen, however, was quite a different animal.

His talk, titled "How to Change the World", was based on his book "Management 3.0" and his work afterwards. I've been interested in the mechanics of change, since I've been trying to change with more or less luck, all the companies I've worked for. Jurgen has a lot to say in that matter, and I've been wrong on so many levels in my attitude towards change, that the talk was very refreshing.

You can take a look at his presentation here:

How to Change the World

Overall, while it's too early to say whether that was the best JVM conference in Poland this year, I can recommend 33rd on all grounds. I feel I can bet blindly on it next year. Grzegorz Duda delivers enormous knowledge in so little time and so little money.

33rd Degree day 2 review

noreply@blogger.com (Jakub Nabrdalik) — Sun, 25 Mar 2012 16:23:00 +0000

Second day of 33rd had no keynotes, and thus was even more intense. A good conference is a conference, where every hour you have a hard dilemma, because there are just too many interesting presentations to see. 33rd was definitely such a conference, and the seconds day really shined.

There were two workshops going on through the day, one about JEE6 and another about parallel programming in Java. I was considering both, but decided to go for presentations instead. Being on the Spring side of the force, I know just as much JEE as I need, and with fantastic GPars (which has Fork/Join, actors, STM , and much more), I won't need to go back to Java concurrency for a while.

GEB - Very Groovy browser automation

Luke Daley works for Gradleware, and apart from being cheerful Australian, he's a commiter to Grails, Spock and a guy behind Geb, a browser automation lib using WebDriver, similar to Selenium a bit (though without IDE and other features).

I have to admit, there was a time where I really hated Selenium. It just felt so wrong to be writing tests that way, slow, unproductive and against the beauty of TDD. For years I've been treating frontend as a completely different animal. Uncle Bob once said at a Ruby conference: "I'll tell you what my solution to frontend tests is: I just don't". But then, you can only go so far with complex GUIs without tests, and once I've started working with Wicket and its test framework, my perspective changed. If Wicked has one thing done right, it's the frontend testing framework. Sure tests are slow, on par with integration tests, but it is way better than anything where the browser has to start up front, and I could finally do TDD with it.

Working with Grails lately, I was more than eager to learn a proper way to do these kind of tests with Groovy.

GEB looks great. You build your own API for every page you have, using CSS selectors, very similar to jQuery, and then write your tests using your own DSL. Sounds a bit complicated, but assuming you are not doing simple HTML pages, this is probably the way to go fast. I'd have to verify that on a project though, since with frontend, too many things look good on paper and than fall out in code.

The presentation was great, Luke managed to answer all the questions and get people interested. On a side note, WebDriver may become a W3C standard soon, which would really easy browser manipulation for us. Apart from thing I expected Geb to have, there are some nice surprises like working with remote browsers (e.g. IE on remote machine), dumping HTML at the end of the test and even making screenshots (assuming you are not working with headless browser).

Micro services - Java, the Unix Way

James Lewis works for ThoughtWorks and gave a presentation, for which alone it was worth to go to Krakow. No, seriously, that was a gem I really didn't see coming. Let me explain what it was about and then why it was such a mind-opener.

ThoughtWorks had a client, a big investment bank, lots of cash, lots of requirements. They spent five weeks getting the analysis done on the highest possible level, without getting into details yet (JEDI: just enough design initially). The numbers were clear: it was enormous, it will take them forever to finish, and what's worse, requirements were contradictory. The system had to have all three guarantees of the CAP theorem, a thing which is PROVED to be impossible.

So how do you deal with such a request? Being ThoughtWorks you probably never say "we can't", and having an investment bank for a client, you already smell the mountains of freshly printed money. This isn't something you don't want to try, it's just scary and challenging as much as it gets.

And then, looking at the requirements and drawing initial architecture, they've reflected, that there is a way to see the light in this darkness, and not to end up with one, monstrous application, which would be hard to finish and impossible to maintain. They have analyzed flows of data, and came up with an idea.

What if we create several applications, each so small, that you can literally "fit it in your head", each communicating with a simple web protocol (Atom), each doing one thing and one thing only, each with it's own simple embedded web server, each working on it's own port, and finding out other services through some location mechanism. What if we don't treat the web as an external environment for our application, but instead build the system as if it was inside the web, with the advantages of all the web solutions, like proxies, caches, just adding a small queue before each service, to be able to turn it off and on, without loosing anything. And we could even use a different technology, with different pair of CAP guarantees, for each of those services/applications.

Now let me tell you why it's so important for me.

If you read this blog, you may have noticed the subtitle "fighting chaos in the Dark Age of Technology". It's there, because for my whole IT life I've been pursuing one goal: to be able to build things, that would be easy to maintain. Programming is a pure pleasure, and as long as you stay near the "hello world" kind of complexity, you have nothing but fun. If we ever feel burned out, demotivated or puzzled, it's when our systems grow so much, that we can no longer understand what's going on. We lose control. And from that point, it's usually just a way downward, towards complete chaos and pain.

All the architecture, all the ideas, practices and patterns, are there for just this reason - to move the border of complexity further, to make the size of "possible to fit in your head" larger. To postpone going into chaos. To bring order and understanding into our systems.

And that really works. With TDD, DDD, CQRS I can build things which are larger in terms of features, and simpler in terms of complexity. After discovering and understanding the methods (XP, Scrum/Kanbad) my next mental shift came with Domain Driven Design. I've learned the building block, the ideas and the main concept of Bounded Contexts. And that you can and should use a different architecture/tools for each of them, simplifying the code with the usage patterns of that specific context in your ming.

That has changed a lot in my life. No longer I have to choose one database, one language and one architecture for the whole application. I can divide and conquer, choose what I want to sacrifice and what advantages I want here, in this specific place of my app, not worrying about other places where it won't fit.

But there is one problem in here: the limit of technologies I'm using, to keep the system simple, and not require omnipotence to be able to maintain, to fix bugs or implement Change Requests.

And here is the accidental solution, ThoughtWorks' micro services bring: if you system is build of the web, of small services that do one thing only, and communicate through simple protocol (like Atom), there is little code to understand, and in case of bugs or Change Requests, you can just tear down one of the services. and build it anew.

James called that "Small enough to throw them away. Rewrite over maintain". Now, isn't that a brilliant idea? Say you have a system like that, build over seven years ago, and you've got a big bag of new requests from your client. Instead of re-learning old technologies, or paying extra effort to try to bring them up-to-date (which is often simply impossible), you decide which services you are going to rewrite using the best tools of your times, and you do it, never having to dig into the original code, except for specification tests.

Too good to be true? Well, there are caveats. First, you need DevOps in your teams, to get the benefits of the web inside your system, and to build in the we as opposite to against it. Second, integration can be tricky. Third, there is not enough of experience with this architecture, to make it safe. Unless... unless you realize, that UNIX was build this way, with small tools and pipes.

That, perhaps. is the best recommendation possible.

Concurrency without Pain in Pure Java

Throughout the whole conference, Grzegorz Duda had a publicly accessible wall, with sticky notes and two sides: what's bad and what's good. One of the note on the "bad" side was saying: "Sławek Sobótka and Paweł Lipiński at the same time? WTF?".

I had the same thought. I wanted to see both. I was luckier though, since I'm pretty sure I'll yet be able too see their presentations this year, as 33rd is the first conference in a long run of conferences planned for 2012. Not being able to decide which one to see, I've decided to go for Venkat Subramaniam and his talk about concurrency. Unless we are lucky at 4Developers, we probably won't see Venkat again this year.

Unfortunately for me, the talk ("show" seems like a more proper word), was very basic, and while very entertaining, not deep enough for me. Venkat used Closure STM to show how bad concurrency is in pure Java, and how easy it is with STM. What can I say, it's been repeated so often, it's kind of obvious by now.

Venkat didn't have enough time to show the Actor model in Java. That's sad, as the further his talk, the more interesting it was. Perhaps there should be a few 90min sessions next year?

Smarter Testing with Spock

After the lunch, I had a chance to go for Sławek Sobótka again, but this time I've decided to listen to one of the commiters of Spock, the best thing in testing world since Mockito.

Not really convinced? Gradle is using Spock (not surprisingly), Spring is starting to use Spock. I've had some experience with Spock, and it was fabulous. We even had a Spock workshop at TouK, lately. I wanted to see what Luke Daley can teach me in an hour.

That was a time well spent. Apart from things I knew already, Luke explained how to share state between tests (@Shared), how to verify exceptions (thrown()), keep old values of variables (old()), how to parametrize description with @Unroll and #parameterName, how to set up data from db or whatever with <<, and a bit more advanced trick with mocking mechanism. Stubbing with closures was especially interesting.

What's new in Groovy 2.0?

Guillaume Laforge is the project lead of Groovy and his presentation was the opposite to what we could see earlier about next versions of Java. Most visible changes were already done in 1.8, with all the AST transformations, and Guillaume spent some time re-introducing them, but then he moved to 2.0, and here apart from multicatch in "throw", the major thing is static compilation and type checking.

We are in the days, were the performance difference between Java and Groovy falls to a mere 20%. That's really little compared to where it all started from (orders of magnitude). That's cool. Also, after reading some posts and successful stories about Groovy++ use, I'd really like to try static compilation with this language

Someone from the audience asked a good question. Why not use Groovy++ as the base for static compilation instead. It turned out that Groovy++ author was also there. The main reason Guillaume gave, were small differences in how they want to handle internal things. If static compilation works fine with 2.0, Groovy++ may soon die, I guess.

Scala for the Intrigued

For the last talk this day, I've chosen a bit of Scala, by Venkat Subramaniam. That was unfortunately a completely basic introduction, and after spending 15 minutes listening about differences between var and val, I've left to get prepared to the BOF session, which I had with Maciek Próchniak.

BOF: Beautiful failures

I'm not in the position to review my own talk, and conclude whether it's failure was beautiful or not, but there is one things I've learned from it.

Never, under none circumstances, never drink five coffees the day you give a talk. To keep my mind active without being overwhelmed by all the interesting knowledge, I drank those five coffees, and to my surprise, when the talk started, the adrenaline shot brought me over the level, where you loose your breath, your pulse, and you start to loose control over your own voice. Not a really nice experience. I've had the effects of caffeine intoxication for the next two days. Lesson learned, I'm staying away from black beans for some time.
If you want the slides, you can find them here.

And that was the end of the day. We went to the party, to the afterparty, we got drunk, we got the soft-reset of our caches, and there came another day of the conference.

You can find my review from the last day in here.

33rd Degree day 1 review

noreply@blogger.com (Jakub Nabrdalik) — Sat, 24 Mar 2012 14:43:00 +0000

33rd Degree is over. After the one last year, my expectations were very high, but Grzegorz Duda once again proved he's more than able to deliver. With up to five tracks (most of the time: four presentations + one workshop), and ~650 attendees, there was a lot to see and a lot to do, thus everyone will probably have a little bit different story to tell. Here is mine.

Twitter: From Ruby on Rails to the JVM

The conference started with Raffi Krikorian from Twitter, talking about their use for JVM. Twitter was build with Ruby but with their performance management a lot of the backend was moved to Scala, Java and Closure. Raffi noted, that for Ruby programmers Scala was easier to grasp than Java, more natural, which is quite interesting considering how many PHP guys move to Ruby these days because of the same reasons. Perhaps the path of learning Jacek Laskowski once described (Java -> Groovy -> Scala/Closure) may be on par with PHP -> Ruby -> Scala. It definitely feels like Scala is the holy grail of languages these days.

Raffi also noted, that while JVM delivered speed and a concurrency model to Twitter stack, it wasn't enough, and they've build/customized their own Garbage Collector. My guess is that Scala/Closure could also be used because of a nice concurrency solutions (STM, immutables and so on).

Raffi pointed out, that with the scale of Twitter, you easily get 3 million hits per second, and that means you probably have 3 edge cases every second. I'd love to learn listen to lessons they've learned from this.

Complexity of Complexity

The second keynote of the first day, was Ken Sipe talking about complexity. He made a good point that there is a difference between complex and complicated, and that we often recognize things as complex only because we are less familiar with them. This goes more interesting the moment you realize that the shift in last 20 years of computer languages, from the "Less is more" paradigm (think Java, ASM) to "More is better" (Groovy/Scala/Closure), where you have more complex language, with more powerful and less verbose syntax, that is actually not more complicated, it just looks less familiar.

So while 10 years ago, I really liked Java as a general purpose language for it's small set of rules that could get you everywhere, it turned out that to do most of the real world stuff, a lot of code had to be written. The situation got better thanks to libraries/frameworks and so on, but it's just patching. New languages have a lot of stuff build into, which makes their set of rules and syntax much more complex, but once you get familiar, the real world usage is simple, faster, better, with less traps laying around, waiting for you to fall.

Ken also pointed out, that while Entity Service Bus looks really simple on diagrams, it's usually very difficult and complicated to use from the perspective of the programmer. And that's probably why it gets chosen so often - the guys selling/buying it, look no deeper than on the diagram.

Pointy haired bosses and pragmatic programmers: Facts and Fallacies of Software Development

Dima got lucky. Or maybe not.

Venkat Subramaniam is the kind of a speaker that talk about very simple things in a way, which makes everyone either laugh or reflect. Yes, he is a showman, but hey, that's actually good, because even if you know the subject quite well, his talks are still very entertaining.

This talk was very generic (here's my thesis: the longer the title, the more generic the talk will be), interesting and fun, but at the end I'm unable to see anything new I'd have learned, apart from the distinction between Dynamic vs Static and Strong vs Weak typing, which I've seen the last year, but managed to forgot. This may be a very interesting argument for all those who are afraid of Groovy/Ruby, after bad experience with PHP or Perl.

Build Trust in Your Build to Deployment Flow!

Frederic Simon talked about DevOps and deployment, and that was a miss in my schedule, because of two reasons. First, the talk was aimed at DevOps specifically, and while the subject is trendy lately, without big-scale problems, deployment is a process I usually set up and forget about. It just works, mostly because I only have to deal with one (current) project at a time.

Not much love for Dart.

Second, while Frederic has a fabulous accent and a nice, loud voice, he tends to start each sentence loud and fade the sound at the end. This, together with mics failing him badly, made half of the presentation hard to grasp unless you were sitting in the first row.

I'm not saying the presentation was bad, far from it, it just clearly wasn't for me.

I've left a few minutes before the end, to see how many people came to Dart presentation by Mike West. I was kind of interested, since I'm following Warsaw Google Technology User Group and heard a few voices about why I should pay attentions to that new Google language. As you can see from the picture on the right, the majority tends to disagree with that opinion.

Non blocking, composable reactive web programming with Iteratees

Sadek Drobi's talk about Iteratees in Play 2.0 was very refreshing. Perhaps because I've never used Play before, but the presentation was flawless, with well explained problems, concepts and solutions.

Sadek started with a reflection on how much CPU we waste waiting for IO in web development, then moved to Play's Iteratees, to explain the concept and implementation, which while very different from the that overused Request/Servlet model, looked really nice and simple. I'm not sure though, how much the problem is present when you have a simple service, serving static content before your app server. Think apache (and faster) before tomcat. That won't fix the upload/download issue though, which is beautifully solved in Play 2.0

The Future of the Java Platform: Java SE 8 & Beyond

Simon Ritter is an intriguing fellow. If you take a glance at his work history (AT&T UNIX System Labs -> Novell -> Sun -> Oracle), you can easily see, he's a heavy weight player.

His presentation was rich in content, no corpo-bullshit. He started with a bit of history of JCP and how it looks like right now, then moved to the most interesting stuff, changes. Now I could give you a summary here, but there is really no point: you'd be much better taking look at the slides. There are only 48 of them, but everything is self-explanatory.

To Java SE 8 and Beyond! | Simon Ritter

While I'm very disappointed with the speed of changes, especially when compared to the C# world, I'm glad with the direction and the fact that they finally want to BREAK the compatibility with the broken stuff (generics, etc.). Moving to other languages I guess I won't be the one to scream "My god, finally!" somewhere in 2017, though. All the changes together look very promising, it's just that I'd like to have them like... now? Next year max, not near the heat death of the universe.

Simon also revealed one of the great mysteries of Java, to me:

The original idea behind JNI was to make it hard to write, to discourage people form using it.

On a side note, did you know Tegra3 has actually 5 cores? You use 4 of them, and then switch to the other one, when you battery gets low.

BOF: Spring and CloudFoundry

Having most of my folks moved to see "Typesafe stack 2.0" fabulously organized by Rafał Wasilewski and Wojtek Erbetowski (with both of whom I had a pleasure to travel to the conference) and knowing it will be recorded, I've decided to see what Josh Long has to say about CloudFoundry, a subject I find very intriguing after the de facto fiasco of Google App Engine.

The audience was small but vibrant, mostly users of Amazon EC2, and while it turned out that Josh didn't have much, with pricing and details not yet public, the fact that Spring Source has already created their own competition (Could Foundry is both an Open Source app and a service), takes a lot from my anxiety.

For the review of the second day of the conference, go here.

Beautiful Failures at 33rd Degree

noreply@blogger.com (Jakub Nabrdalik) — Mon, 19 Mar 2012 13:35:00 +0000

33rd in Kraków, is rolling baby.

Tomorrow, together with Maciek Próchniak, we are giving a talk about failures.

There is a problem with failures withing our culture, and by our, I mean central and eastern Europe. In San Fransisco, there are regular meetings, called Mobile Monday, where speakers start by saying how many start ups they have failed, and it’s been seen as a their reference to wisdom. At the very end, they’ve learned a lot from all this failures. And it’s not limited only to San Francisco or Mobile Monday. It’s their culture, every failure gets you smarter. In US it's OK to fail.

Have you heard a story about a Japanese train controller that committed suicide, when two trains in a row have been late because of his mistakes? The Europe may not be that extreme, but it’s still at least inappropriate to admit, that you ever made a mistake.

If we never admit, if we never reflect, we never learn. So we are changing the rules for an hour. There is nothing good or bad without a context, and we would like to share the circumstances under which things don’t work.

Things like:

shared responsibility
self organized teams
gamification
open source
metaprogramming
‘enterprise’ technologies

See yoy there.

How to get to TouK

noreply@blogger.com (Jakub Nabrdalik) — Mon, 23 Jan 2012 20:08:00 +0000

This Tuesday, architect-wannabe group has it's meeting at TouK at 18:00. Since TouK is not an easy company to find, here's the video tutorial Piotr Przybylak asked for. It starts at Rondo Zesłańców Syberyjskich (driving from north). Hope it helps.

Thanks to Maciej Próchniak for participation :)

And here is a map:

Show larger map

Bash'ing your git deployment

noreply@blogger.com (Jakub Nabrdalik) — Thu, 12 Jan 2012 14:22:00 +0000

Chuck Norris deploys after every commit. Smart men deploy after every successful build on their Continuous Integration server. Educated men, deploy code directly from their distributed version control systems. I, being neither, had to write my deployment script in bash.

We're using git and while doing so I wanted us to:

deploy from working copy, but...
make sure that you can deploy only if you committed everything
make sure that you can deploy only if you pushed everything upstream
tag the deployed hash
display changelog (all the commits between two last tags)

Here are some BASH procedures I wrote on the way, if you need them:

make sure that you can deploy only if you committed everything

verifyEverythingIsCommited() {
    gitCommitStatus=$(git status --porcelain)
    if [ "$gitCommitStatus" != "" ]; then
        echo "You have uncommited files."
        echo "Your git status:"
        echo $gitCommitStatus
        echo "Sorry. Rules are rules. Aborting!"
        exit 1
    fi
}

make sure that you can deploy only if you pushed everything upstream

verifyEverythingIsPushedToOrigin() {
    gitPushStatus=$(git cherry -v)
    if [ "$gitPushStatus" != "" ]; then
        echo "You have local commits that were NOT pushed."
        echo "Your 'git cherry -v' status:"
        echo $gitPushStatus
        echo "Sorry. Rules are rules. Aborting!"
        exit 1
    fi
}

tag the deployed hash

Notice: my script takes first parameter as the name of the server to deploy to (this is $1 passed to this procedure). Also notice, that 'git push' without the '--tags' does not push your tags.

tagLastCommit() {
    d=$(date '+%y-%m-%d_%H-%M-%S')
    git tag "$1_$d"
    git push --tags
}

This creates nice looking tags like these:
preprod_12-01-11_15-16-24
prod_12-01-12_10-51-33
test_12-01-11_15-11-10
test_12-01-11_15-53-42

display changelog (all the commits between two last tags)

printChangelog() {
    echo "This is changelog since last deploy. Send it to the client."
    twoLastHashesInOneLine=$(git show-ref --tags -s | tail -n 2 | tr "\\n" "-");
    twoLastHashesInOneLineWithThreeDots=${twoLastHashesInOneLine/-/...};
    twoLastHashesInOneLineWithThreeDotsNoMinusAtTheEnd=$(echo $twoLastHashesInOneLineWithThreeDots | sed 's/-$//');
    git log --pretty=oneline --no-merges --abbrev-commit  $twoLastHashesInOneLineWithThreeDotsNoMinusAtTheEnd
}

The last command gives you a nice log like this:
e755c63 deploy: fix for showing changelog from two first tags instead of two last ones
926eb02 pringing changelog between last two tags on deployment
34478b2 added git tagging to deploy

Atom Feeds with Spring MVC

noreply@blogger.com (Jakub Nabrdalik) — Thu, 12 Jan 2012 13:24:00 +0000

How to add feeds (Atom) to your web application with just two classes?
How about Spring MVC?

Here are my assumptions:

you are using Spring framework
you have some entity, say “News”, that you want to publish in your feeds
your "News" entity has creationDate, title, and shortDescription
you have some repository/dao, say "NewsRepository", that will return the news from your database
you want to write as little as possible
you don't want to format Atom (xml) by hand

You actually do NOT need to use Spring MVC in your application already. If you do, skip to step 3.

Step 1: add Spring MVC dependency to your application
With maven that will be:

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>3.1.0.RELEASE</version>
</dependency>

Step 2: add Spring MVC DispatcherServlet
With web.xml that would be:

<servlet>
    <servlet-name>dispatcher</servlet-name>
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    <init-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>classpath:spring-mvc.xml</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
    <servlet-name>dispatcher</servlet-name>
    <url-pattern>/feed</url-pattern>
</servlet-mapping>

Notice, I set the url-pattern to “/feed” which means I don't want Spring MVC to handle any other urls in my app (I'm using a different web framework for the rest of the app). I also give it a brand new contextConfigLocation, where only the mvc configuration is kept.

Remember that, when you add a DispatcherServlet to an app that already has Spring (from ContextLoaderListener for example), your context is inherited from the global one, so you should not create beans that exist there again, or include xml that defines them. Watch out for Spring context getting up twice, and refer to spring or servlet documentation to understand what's happaning.

Step 3. add ROME – a library to handle Atom format
With maven that is:

<dependency>
    <groupId>net.java.dev.rome</groupId>
    <artifactId>rome</artifactId>
    <version>1.0.0</version>
</dependency>

Step 4. write your very simple controller

@Controller
public class FeedController {
    static final String LAST_UPDATE_VIEW_KEY = "lastUpdate";
    static final String NEWS_VIEW_KEY = "news";
    private NewsRepository newsRepository;
    private String viewName;

    protected FeedController() {} //required by cglib

    public FeedController(NewsRepository newsRepository, String viewName) {
        notNull(newsRepository); hasText(viewName);
        this.newsRepository = newsRepository;
        this.viewName = viewName;
    }

    @RequestMapping(value = "/feed", method = RequestMethod.GET)        
    @Transactional
    public ModelAndView feed() {
        ModelAndView modelAndView = new ModelAndView();
        modelAndView.setViewName(viewName);
        List<News> news = newsRepository.fetchPublished();
        modelAndView.addObject(NEWS_VIEW_KEY, news);
        modelAndView.addObject(LAST_UPDATE_VIEW_KEY, getCreationDateOfTheLast(news));
        return modelAndView;
    }

    private Date getCreationDateOfTheLast(List<News> news) {
        if(news.size() > 0) {
            return news.get(0).getCreationDate();
        }
        return new Date(0);
    }
}

And here's a test for it, in case you want to copy&paste (who doesn't?):

@RunWith(MockitoJUnitRunner.class)
public class FeedControllerShould {
    @Mock private NewsRepository newsRepository;
    private Date FORMER_ENTRY_CREATION_DATE = new Date(1);
    private Date LATTER_ENTRY_CREATION_DATE = new Date(2);
    private ArrayList<News> newsList;
    private FeedController feedController;

    @Before
    public void prepareNewsList() {
        News news1 = new News().title("title1").creationDate(FORMER_ENTRY_CREATION_DATE);
        News news2 = new News().title("title2").creationDate(LATTER_ENTRY_CREATION_DATE);
        newsList = newArrayList(news2, news1);
    }

    @Before
    public void prepareFeedController() {
        feedController = new FeedController(newsRepository, "viewName");
    }

    @Test
    public void returnViewWithNews() {
        //given
        given(newsRepository.fetchPublished()).willReturn(newsList);
        
        //when
        ModelAndView modelAndView = feedController.feed();
        
        //then
        assertThat(modelAndView.getModel())
                .includes(entry(FeedController.NEWS_VIEW_KEY, newsList));
    }

    @Test
    public void returnViewWithLastUpdateTime() {
        //given
        given(newsRepository.fetchPublished()).willReturn(newsList);

        //when
        ModelAndView modelAndView = feedController.feed();

        //then
        assertThat(modelAndView.getModel())
                .includes(entry(FeedController.LAST_UPDATE_VIEW_KEY, LATTER_ENTRY_CREATION_DATE));
    }

    @Test
    public void returnTheBeginningOfTimeAsLastUpdateInViewWhenListIsEmpty() {
        //given
        given(newsRepository.fetchPublished()).willReturn(new ArrayList<News>());

        //when
        ModelAndView modelAndView = feedController.feed();

        //then
        assertThat(modelAndView.getModel())
                .includes(entry(FeedController.LAST_UPDATE_VIEW_KEY, new Date(0)));
    }
}

Notice: here, I'm using fest-assert and mockito. The dependencies are:

<dependency>
 <groupId>org.easytesting</groupId>
 <artifactId>fest-assert</artifactId>
 <version>1.4</version>
 <scope>test</scope>
</dependency>
<dependency>
 <groupId>org.mockito</groupId>
 <artifactId>mockito-all</artifactId>
 <version>1.8.5</version>
 <scope>test</scope>
</dependency>

Step 5. write your very simple view
Here's where all the magic formatting happens. Be sure to take a look at all the methods of Entry class, as there is quite a lot you may want to use/fill.

import org.springframework.web.servlet.view.feed.AbstractAtomFeedView;
[...]

public class AtomFeedView extends AbstractAtomFeedView {
    private String feedId = "tag:yourFantastiSiteName";
    private String title = "yourFantastiSiteName: news";
    private String newsAbsoluteUrl = "http://yourfanstasticsiteUrl.com/news/"; 

    @Override
    protected void buildFeedMetadata(Map<String, Object> model, Feed feed, HttpServletRequest request) {
        feed.setId(feedId);
        feed.setTitle(title);
        setUpdatedIfNeeded(model, feed);
    }

    private void setUpdatedIfNeeded(Map<String, Object> model, Feed feed) {
        @SuppressWarnings("unchecked")
        Date lastUpdate = (Date)model.get(FeedController.LAST_UPDATE_VIEW_KEY);
        if (feed.getUpdated() == null || lastUpdate != null || lastUpdate.compareTo(feed.getUpdated()) > 0) {
            feed.setUpdated(lastUpdate);
        }
    }

    @Override
    protected List<Entry> buildFeedEntries(Map<String, Object> model, HttpServletRequest request, HttpServletResponse response) throws Exception {
        @SuppressWarnings("unchecked")
        List<News> newsList = (List<News>)model.get(FeedController.NEWS_VIEW_KEY);
        List<Entry> entries = new ArrayList<Entry>();
        for (News news : newsList) {
            addEntry(entries, news);
        }
        return entries;
    }

    private void addEntry(List<Entry> entries, News news) {
        Entry entry = new Entry();
        entry.setId(feedId + ", " + news.getId());
        entry.setTitle(news.getTitle());
        entry.setUpdated(news.getCreationDate());
        entry = setSummary(news, entry);
        entry = setLink(news, entry);
        entries.add(entry);
    }

    private Entry setSummary(News news, Entry entry) {
        Content summary = new Content();
        summary.setValue(news.getShortDescription());
        entry.setSummary(summary);
        return entry;
    }

    private Entry setLink(News news, Entry entry) {
        Link link = new Link();
        link.setType("text/html");
        link.setHref(newsAbsoluteUrl + news.getId()); //because I have a different controller to show news at http://yourfanstasticsiteUrl.com/news/ID
        entry.setAlternateLinks(newArrayList(link));
        return entry;
    }

}

Step 6. add your classes to your Spring context
I'm using xml approach. because I'm old and I love xml. No, seriously, I use xml because I may want to declare FeedController a few times with different views (RSS 1.0, RSS 2.0, etc.).

So this is the forementioned spring-mvc.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
        <property name="mediaTypes">
            <map>
                <entry key="atom" value="application/atom+xml"/>
                <entry key="html" value="text/html"/>
            </map>
        </property>
        <property name="viewResolvers">
            <list>
                <bean class="org.springframework.web.servlet.view.BeanNameViewResolver"/>
            </list>
        </property>
    </bean>

    <bean class="eu.margiel.pages.confitura.feed.FeedController">
        <constructor-arg index="0" ref="newsRepository"/>
        <constructor-arg index="1" value="atomFeedView"/>
    </bean>

    <bean id="atomFeedView" class="eu.margiel.pages.confitura.feed.AtomFeedView"/>
</beans>

And you are done.

I've been asked a few times before to put all the working code in some public repo, so this time it's the other way around. I've describe things that I had already published, and you can grab the commit from the bitbucket.

Hope that helps.

JBoss Envers and Spring transaction managers

noreply@blogger.com (Jakub Nabrdalik) — Tue, 20 Sep 2011 15:56:00 +0000

I've stumbled upon a bug with my configuration for JBoss Envers today, despite having integration tests all over the application. I have to admit, it casted a dark shadow of doubt about the value of all the tests for a moment. I've been practicing TDD since 2005, and frankly speaking, I should have been smarter than that.

My fault was simple. I've started using Envers the right way, with exploratory tests and a prototype. Then I've deleted the prototype and created some integration tests using in-memory H2 that looked more or less like this example:

@Test
public void savingAndUpdatingPersonShouldCreateTwoHistoricalVersions() {
    //given
    Person person = createAndSavePerson();
    String oldFirstName = person.getFirstName();
    String newFirstName = oldFirstName + "NEW";

    //when
    updatePersonWithNewName(person, newFirstName);

    //then
    verifyTwoHistoricalVersionsWereSaved(oldFirstName, newFirstName);
}

private Person createAndSavePerson() {
    Transaction transaction = session.beginTransaction();
    Person person = PersonFactory.createPerson();
    session.save(person);
    transaction.commit();
    return person;
}    

private void updatePersonWithNewName(Person person, String newName) {
    Transaction transaction = session.beginTransaction();
    person.setFirstName(newName);
    session.update(person);
    transaction.commit();
}

private void verifyTwoHistoricalVersionsWereSaved(String oldFirstName, String newFirstName) {
    List<Object[]> personRevisions = getPersonRevisions();
    assertEquals(2, personRevisions.size());
    assertEquals(oldFirstName, ((Person)personRevisions.get(0)[0]).getFirstName());
    assertEquals(newFirstName, ((Person)personRevisions.get(1)[0]).getFirstName());
}

private List<Object[]> getPersonRevisions() {
    Transaction transaction = session.beginTransaction();
    AuditReader auditReader = AuditReaderFactory.get(session);
    List<Object[]> personRevisions = auditReader.createQuery()
            .forRevisionsOfEntity(Person.class, false, true)
            .getResultList();
    transaction.commit();
    return personRevisions;
}

Because Envers inserts audit data when the transaction is commited (in a new temporary session), I thought I have to create and commit the transaction manually. And that is true to some point.

My fault was that I didn't have an end-to-end integration/acceptance test, that would call to entry point of the application (in this case a service which is called by GWT via RPC), because then I'd notice, that the Spring @Transactional annotation, and calling transaction.commit() are two, very different things.

Spring @Transactional annotation will use a transaction manager configured for the application. Envers on the other hand is used by subscribing a listener to hibernate's SessionFactory like this:

<bean id="sessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean" >        
...
 <property name="eventListeners">
     <map key-type="java.lang.String" value-type="org.hibernate.event.EventListeners">
         <entry key="post-insert" value-ref="auditEventListener"/>
         <entry key="post-update" value-ref="auditEventListener"/>
         <entry key="post-delete" value-ref="auditEventListener"/>
         <entry key="pre-collection-update" value-ref="auditEventListener"/>
         <entry key="pre-collection-remove" value-ref="auditEventListener"/>
         <entry key="post-collection-recreate" value-ref="auditEventListener"/>
     </map>
 </property>
</bean>

<bean id="auditEventListener" class="org.hibernate.envers.event.AuditEventListener" />

Envers creates and collects something called AuditWorkUnits whenever you update/delete/insert audited entities, but audit tables are not populated until something calls AuditProcess.beforeCompletion, which makes sense. If you are using org.hibernate.transaction.JDBCTransaction manually, this is called on commit() when notifying all subscribed javax.transaction.Synchronization objects (and enver's AuditProcess is one of them).

The problem was, that I used a wrong transaction manager.

<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager" >
    <property name="dataSource" ref="dataSource"/>
</bean>

This transaction manager doesn't know anything about hibernate and doesn't use org.hibernate.transaction.JDBCTransaction. While Synchronization is an interface from javax.transaction package, DataSourceTransactionManager doesn't use it (maybe because of simplicity, I didn't dig deep enough in org.springframework.jdbc.datasource), and thus Envers works fine except not pushing the data to the database.

Which is the whole point of using Envers.

Use right tools for the task, they say. The whole problem is solved by using a transaction manager that is well aware of hibernate underneath.

<bean id="transactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager" >
    <property name="sessionFactory" ref="sessionFactory"/>
</bean>

Lesson learned: always make sure your acceptance tests are testing the right thing. If there is a doubt about the value of your tests, you just don't have enough of them,

NoSQL devmeeting in Warsaw

noreply@blogger.com (Jakub Nabrdalik) — Sun, 11 Sep 2011 10:34:00 +0000

I've spent this Saturday at NoSQL devmeeting in Warsaw, organized by Adam Lider, Piotr Zwoliński and lead by David de Rosier. At first I was reluctant to go, as my level of js mastery is clearly negative, and I have only theoretical knowledge about NoSQL databases, but as Maciej Próchniak noticed, these are exactly the reasons why I should.

The meeting started at 9am and lasted till 9pm (though I had to leave at 7pm), with one-hour break for lunch. David began with a gentle but fantastic introduction to CouchDB, MongoDB, Cassandra and Redis, after which we were split into groups of 3-4, each taking one of the databases aforementioned. Our task was simple: with four big MySQL (partitioned) dumps on the local SVN, we were to migrate the data to our NoSQL DB and then prepare a simple twitter application in any language we want, preferably javascript using Node.js.

Our group took the hard way of playing with Node.js, easing it up with the choice of MongoDB (as it seems to have the best community and thus support). Node.js was more of an obstacle than help, mainly because of it asynchronous nature, but it's quite possible that we just don't know how to write good code in it. We definitely didn't try much, being fine with “hey, it works!”, which is just right for the kind of hacking/prototyping we were into.

We had no problems with MongoDB. With Barack Obama's 9 million followers in mind, we settled on the best model early, choosing eventual consistency and data duplication in the name of simplicity and query performance.

Because I had to go two hours early, I've missed the part where we were to test the performance and try our luck with replication, but nonetheless this Saturday was clearly awesome. I loved the hackengarded at 33rd Degree, but devmeetings are even more fabulous. The formula is superb. If I were to add anything, it would be an open review of each application at the end. I'm really curious how other teams did it.

Checkout devmeetings web page and if you have a chance to visit one, definitely go for it. It's great, it's free, and it's one of the best ways to learn something useful fast.

How to write a good Request for Proposal

noreply@blogger.com (Jakub Nabrdalik) — Mon, 05 Sep 2011 11:47:00 +0000

For the last month and a half, I've been busy writing offers in reply to Requests for Proposal (RFP) from many different companies: startups, big corporations and everything in between. Since the level of those RFPs vary, and since the quality of RFP always influences the price you get, I'd like to share a few tips, about how to write a good RFP without any knowledge in analysis, and how to get the lowest possible price.

What is a Request for Proposal

Simply put, it's a description of a non-existing system, for which software developers usually answer with a formal offer.

(RFP) The publication by a prospective software
          purchaser of details of the required system in order to
          attract offers by software developers to supply it. Software
          development under contract starts with the selection of the
          software developer by the customer. A request for proposal
          (also called in Britain an "invitation to tender") is the
          beginning of the selection process.

          [Bennatan, E.M., "Software Project Management", 2nd edition,
          McGraw-Hill International, 1992].

Most RFPs require a fixed-price on the offer, because the buyer wants to know how much will it costs to make the system, and be able to compare both the potential value and prices.

For whom it may concern...

Before I get into details, bear in mind, these advices were written for projects up to a few man-years in size/scope. For anything bigger I'd strongly suggest starting with a short analysis as a separate project. Probably done by some other company. Beware of pure consulting companies though – they cost a lot and without the battlefield experience, often fail to create anything worth the money. Sometimes even jeopardizing projects with their choice of poor technologies and architectures. You'll be much safer asking a good software company, which is also able to provide the solution, to do the analysis.

It's a pity but I've seen consultants working against their customers or against developers, for very trivial reason, coming directly from human psychology. Maslow's hierarchy of needs puts “respect by others” before morality, and a typical consultant without up to date battlefield experience, will never be respected by developers. So instead of helping, he'll do anything to make himself feel important, like fighting for features no one needs, or insisting on using SOA where it's completely useless. I've even seen whole projects, created for customers who couldn't understand them, wouldn't need them, and would abandon them right after spending a few men years in money and effort to create them, just because some out-of-place consultant used words like cloud and synergy to charm the CEO.

Beware of anyone who speaks about synergy. Chances are, they already lost their connection with reality. Save yourself a lot of trouble and hire a real software provider to do the analysis instead.

This post may also be useless, if you do not need a fixed-price offer and you have a trusted software company, that will play fair with you. But that's not common, unfortunately.

Why is the quality of RFP significant for the price?

In an ideal world, you just have to say that you need a developer's assistance, and some analyst will come to you to write down what you really want. Even things you are not aware of. In the ideal world, analysts work for free.

But in the real world, resources are limited, and if you need something worth 300 man-days, the company usually doesn't want to risk more than 10 man-days (two people per week) before you decide whether you want to pay or not. 10 man-days may seem like a lot, except when you need to make a prototype, technology reconnaissance, understand the domain and propose valid estimations and formal documents, it's really not.

This is the reason, why the developer often cannot have more than one or two meetings with the client before the offer is finished. This is the reason why the quality of your RFP is crucial

The less informative your request is, the higher the price will be. To understand that, you need to understand where the price on an offer comes from. It usually consist of at least three components:

the effort counted in man-days (function points and so on are finally translated to man-days at the end anyway)
the margin, that the provider has to have, to pay for all the costs not directly connected with the project (this is where you pay for the offer itself)
the uncertainty or fear

The margin depends on many things, being lower mostly for companies that hire more developers than marketing/management people (which explains, why big companies are often so expansive, and why agile software houses like to stay small).

The effort depends on the technology chosen, the internal experience of a company in given technology, the method (SCRUM being cheaper than full RUP with all the bells and whistles, but not that much compared to Open Unified Process, for example) and whatever libraries the company already has on hand.

The uncertainty is where a large part of the price is placed.

See, the company sending a formal offer for a fixed-price project, usually is bond by law, and has no way to change it later. The theory says our estimations at the beginning of the analysis may go wrong up to 400%. I've personally seen a project where the initial estimation of nine man-weeks turned out to be four man-years. Or another one, that turned from 42 man-months into 252. These are big differences that can bring a company down, and to cope with that, software providers have to have an uncertainty margin in their estimations.

And you, as a client, do not want that.

You don't want that, because either the price will be higher than possible (if their effort estimations were right, but risk factor too high), or the provider may go out of business (if estimations were very wrong) in which case you are screwed anyway.

If you think you want to trick them just a bit to get the software for just a little under the cost, think again, because software companies in case of an emergency often try to reduce one thing which wasn't set in stone in the offer and is most difficult to verify: quality.

You see, the famous tetrahedron of cost-scope-time-quality is wrong. It doesn't work that way. Not for software at least.

I'll skip the trivial cost-time phenomena for a moment, we all know that adding more women won't bring the baby faster. Each project has a sweet spot in number of developers, above which the cost rises exponentially. Let me show you something different. Let me explain how quality works.

Most developers usually work on one quality level – their maximum, which of course may be quite low anyway, but is still their top. A good developer will use all the techniques he is proficient with, A/B/TDD, self-documenting code, prototyping etc. because Uncle Bob is right saying, that the only way to go really fast is to go well. And we all hate debugging anyway.

So the developer works at one quality level. Unless someone comes in and tells him to hurry up no matter what. And that someone may be a PM in panic, that can clearly see now, that the initial estimation was wrong, and the only way to not lose too much money is to cut wherever possible.

This is a risky way, short sighted way, but may work if it slips by your acceptance tests. And so you, the client, are going to pay for it later, in change requests. And it works better, the bigger the customer is, the further away the guy accepting features is from the stakeholder and the user.

Which explains why so many IT projects for public administration are totally fucked up.

Some desperate companies even go so far, as to deliberately give you a price under the cost, knowing that they will charge insane amount of money on change requests later. But there is a solution to that: require all the providers to sell you non-obfuscated source code, with automatic tests, and make sure it's clear with them, that you will choose a company supporting the solution later. That should make them think twice.

So you don't want to screw the provider, because at the end you will screw yourself. You want the best deal for your bucks, low support cost and you want to have o good relationship with the developer after the project is done, because trust is the biggest value in every business.

Let's bring the price as low as possible, without loosing anything, by reducing fear and uncertainty with a few simple tips.

Use cases (interaction) instead of lists and screens

You don't need to be a software analyst to be in need of a software solution, and so you may want to describe the idea for the software, that is in your mind, using screens, list of nouns and so on. Analysts have discovered that the best way to write down requirements, is to use User Stories or informal Use-Cases (for the rest of this post, I'll forget about any formal differences between those two).

What are those things? You don't need to read “Writing effective use cases” by Alistair Cockburn (though it is a great book), you just need to remember one thing: describe interactions between the system and everything around it (users and other systems).

Don't think about HOW user interface looks like, think about WHAT the user can do with it.

I'll give you an example. If you write something like this

user management panel

there is a big uncertainty factor, because the implementation of this small otherwise task, may vary by 300%, depending on your actual needs. But if you write something like this:

administrator can delete users, change their roles, change their passwords, search them by email and login, and see their details

my uncertainty drops close to zero, because I know exactly what I need and what I need not to implement.

So write about interactions the system in your mind has with it's users and with other systems. And don't even try to be formal – it's not needed.

No integration without interaction (and protocols)

What do you think when you see something like this

SSO integration

That seems simple. SSO stands for Single Sign On, so we probably need to authenticate the user, right? Too bad that the real requirement looks like this:

the user logs in to the system through our SSO, and if the user wants to register (sign-up), he should also be added through our SSO API with GUI in the portal

Ha! So it's a bit more than logging in. But is it enough to estimate the effort and keep the fear margin close to zero?

No.

Because SSO is just a concept. There is no information about the actual protocol. And there are plenty on the market. Maybe it's Kerberos or NTLM? Or maybe it's OpenID? OAuth2 anybody? Or maybe it's LDAP, and we already have tons of libraries for that, no implementation needed. Or maybe, god forbid, it's some CORBA service that remembers the times when dragons and demons ruled the earth.

This is a simple rule for requesting integration features: write about the INTERACTION (what the system wants to do from the business point of view) and about the PROTOCOL underneath. That's the minimum needed.

And if you have some experience with given system/protocol, write those too, whether it's bad or good. That will help a lot, especially for proprietary technologies. If your lawyers allow for that, send us the docs or a code sample. Nothing is as risky as integration with a third-party that we know nothing about.

Write what you don't know

With all the focus on putting all the knowledge you have about the system in the request, it's just as crucial to put there what you do not know.

When people think about a fixed-price project, they usually think about a single price, but that's not the way it has to be. You may need the fixed-price to be able to compare different offers and make you feel safe, but it doesn't mean, you cannot have options.

There is no way to make all the decisions before the project is started. Agile suggests to postpone making decisions to the last possible moment, because then, you know more and are able to make it more consciously. If you have a few ways to solve a given business problem, and do not see which one is better right away, you can ask us to estimate both solutions. If their costs differ, it may help you to choose. If not, let's leave it for later, just assume one of them is used.

Options are good. We can even prepare a proposal with two or three stages, where with the first one, you'll get your working software with all crucial features, with the second all non-crucial features will be implemented, and we will leave the bells and whistles for the third stage. In fact, these is what we usually do.

We have short cycles, we can get you new features on production every week, since our sprints are week long. Don't be afraid to write down what you don't know, or what you cannot decide yet on. We will prepare a few options and we will have that in mind, when creating the proposal.

No more unidentified TLAs

Even with my limited skills, I really like English, the lingua franca of IT. It's simple, elegant, and everything sounds better in it. But it has one big issue, polluting it from the mid-twenties: acronyms. And our industry is so full of them, it makes you want to Fornicate Under Consent of King.

It's easy to call the customer and get them all right, finally, but do you really want to have so much time wasted (multiply TLAs by number of providers)? Remember, that the guys preparing an offer for you, also have just this much time, and they could spend it on lowering uncertainty/cost of some other part of your project, instead on guessing TLAs. Please, save us and yourself the trouble, and every first time you use a given TLA (three letter acronym), add a full name in the parenthesis.

You know, it's not that big of a problem that we need to spend more time, finding out what you had on your mind, it's just that if there is a mistake, it's going to be painful. And there'll be mistakes, finally, because TLAs overlap, even in given context, and having so many of them, it's statistically unlikely we will always get everything right.

Things get much worse, when the request for proposal mixes two or three languages (which is inevitable, unless the whole document is in English), because on top of English acronyms, we get German acronyms, Polish acronyms, and short names of products from vendors that no longer exists. It's an invitation for error. And you don't want us to be wrong with the estimation.

Ask for CVs: companies are names, people are skills.

Some of our customers ask for Curriculum Vitae for every single person on the project. It's a great idea, because at the very end, it's not a company that creates software, it's the developer. Developers change their jobs quite often, and you may end up hiring a company with good references in the domain, only to find out, that the team is completely fresh to the subject.

It may be as well, the other way around: getting a proposal from a company with absolutely no references, only to find out that it's started by experienced veterans, who left the other software provider, to create their own business.

Skills matter, asking questions costs you nothing, so go ahead and ask for CVs, unless you want to find out that your critical project is created by two containers of first year students, that got hired for a summer internship.

A software house, having a lot of projects going, may not be able to guarantee that the developers in the offer will be there when you finally make up your mind to sign the contract, but should still provide you with potential CVs. And if it won't... well, have a lot of diapers ready because a few containers of freshmen are coming your way.

Meaningful performance expectations

Performance requirements are included in nearly every request for proposal, and that's cool, because more often than not, the possible architecture depends on the traffic (number of messages, calls or users). I don't need to be lightning fast for your 50 Intranet users, and a handful of messages per minute, I can focus on productivity and keep the price low. I may need to use CQRS with a fast no-SQL cache/database, eventual consistency, queues, partitioning and Hadoop with MapReduce, if you want to have bazillions of users per second with possibility of tens-of-bazillions the next year on your existing hardware configuration. And that's a completely different effort.

Or I may be required to present a hardware configuration that will do the trick with response time < 1 sec. Either way I need to know what your expectations are, but I need a bit more than

expected number of users: 50 000

Where are those users from? How is the traffic distributed throughout the day, week and month? What is a typical use case scenario? Finally: what are the expected peaks and how important is the response time for your business?

You may not be able to answer, that's fine, but if you cannot, how do you expect us to guarantee the performance?

Sometimes I read something like: up to 50 requests per second in peaks on SPARC M8000 with 16 processors. These are valid requirements, knowing that those 16 processors have 64 cores together, I know that if I keep the response time at less than a second for a request on one core, I should be fine. And that doesn't seem like a problem event though I can't prove it, because I only have access to Intel's Xeons and the software doesn't exist yet. I can still prepare a little benchmark or a proof-of-concept. Here I keep my risk factor low.

But 50 000 users on three AMD's Opterons? Hell if I know. I've been building sites for 2mln users on that configuration, so it looks fine, but the question is what peaks look like? So I'm going to add a bit to cover that risk, and a few days at least for optimization.

If you want to have the price low in the offer you get, either write the details about peaks or don't ask for a guarantee. Ask for performance and give your expectations, and we'll keep that in mind and do everything to get you that performance, but guarantees are possible only when we can prove something, and more often than not, we can't. Usually it's just a bet, so you'll be paying extra for this risk.

Bird's eye view is nice, details are crucial

Sometimes the system in scope is so large, that the request is for just a part of it. It's a good idea to get your potential providers to understand the big picture, but it's even more important to make it clear, what your expectation as to their job is.

We had a request for a system that looked like a hardware plus embedded software solution. We were going to reject it, not even trying, because we don't do hardware, and our skills with given embedded technology is negligible. But a question remained: why would they even send us this request, knowing very well what we do, and what we don't. Trying to figure it, we found out, that our job would be only to prepare an integration point and all the embedded software would be written by another company, waiting for us to give them an API.

When you are describing the ecosystem the software is going to live in, make sure you are clear as to what a given software house is supposed to create. Otherwise, you may not get an answer at all from some of them, giving you less choice.

How much do you have (for startups only)

With startups, the first question I always ask is: how much money do you have?

Funny, isn't? The whole point of a request for proposal is to find out, how much would the system cost from different providers, and yet I ask for how much they want to spend at the very beginning.

And while I would always like to know it, when preparing an offer, I'm only asking startups.

Why?

Because, existing companies are more flexible with their money. The software is there to help them achieve their business goal and if they need to invest more, to get more, they may go for it. Startups usually don't have that choice. Startups have an idea, that will hopefully bring tons of money, and they have some initial funding, that has to be enough.

So I need to know, what part of the funding is for IT, so that I can find a solution that may be acceptable. If the money seems like no-way for all the requirements, maybe we can split them into two: one part to get the business going, the other, when you finally start making some money. It's easier to find additional funding, when you can at least cover your costs. Or perhaps, we need to change a non-crucial part of your business idea for the first year, the cut the cost dramatically, and get you going. Maybe we can get you an off-the-shelf open source system for the parts that do not build your market advantage on. There ale plenty of possibilities.

I've also seen a few lucky startups, that couldn't invest too little in the IT of their business, because the investor wasn't interested in spending a mere few hundred thousand, to get a million back. The investor wanted to spend a few million to get back a few hundred million.

That's cool with me, but let me know from the start, because then, I'll go for solutions better fitting your capabilities.

Call me / email me if you can't write it formally

Frankly speaking, if I was the owner of the software house responding to requests for proposal, I wouldn't have balls to sign half the contracts we do. At least with corporations. Most of the requests have ugly sections, from even uglier lawyers, that generally require you to sell your soul and your wife, in case the other side requires it.

You know how long one can negotiate a Non-Disclosure Agreement, before finally getting the request for proposal? A month. That's how long it takes, before the lawyer finally understands, that he cannot prohibit us from working for ALL THE OTHER companies in the world, just because he wishes so.

Anyway, apart from legal formalities, corporations often have their request templates polluted with all sorts of smelly stuff, that implies a starting cost.

For example, an Intranet application template is being used for a public Internet application, and as a result we are asked to prepare a formal end-user documentation in doc format. Seriously? Come on, find me a single user that would download and read a doc before using the site. Can we get you a context help instead (which we will implement anyway)?

And the answer is: no ;)

No ;) Means we, the corporation, need a formal doc document because this is the way some consulting company (gaaah!) years ago decided, and charged us a million for preparing these templates, and it's not in my power to change that, but ;) also means nobody's going to read it, and you can produce those docs from the context help, if that brings the price down. And yes, in my private opinion, context help will be of much better use than a doc.

Same story with RUP. Do you guys really need that? Is Open Unified Process (which adopts the core of RUP, while being more agile) enough? Sometimes you do not care, it's just the template that says RUP.

The easiest way to keep the price low is to call us. Tell us what you really don't care about, and what is there just because of the consulting company.

I'd be scared to sign a document which obliges me to formal things I won't do (I've done that once, and now I have to, theoretically: “consult all important life decisions with the bank”), but my bosses have more experience in that matter (or just cojones) and understanding with our clients, for whom we've been working from their very beginning.

Everything changes (be agile or be dead)

It's quite embarrassing, that even in 2011, some companies literally stick to the offer. It may be because of fear and the potential legal consequences, but this is not the way it should be and it's not the way we work.

If you do not play dirty, good software houses will always welcome change. Let us make decision whether we can handle it within our budget, and if we can, we won't bother you with an official change request (CR) or invoice/agreement for every change. If we won't, let us find a cheaper way, to meet your business needs. We are here to cooperate, and we want to have you happy at the end of the project, because we want you to trust us. We are playing on the same side. At the end of the day, we want you to flourish, and we want to sent us another request for proposal, when you'll be in a need of another software solution.

We will make you secure, by selling you well written sources and using open-sourced libraries, by creating tons of tests, so that another company can easily take over the support if you wish so.

This is all old news, Open Source movement and Agile movement, have been here so long it should be crystal clear, it's better to play together this way.

If you have a negative experience with a company over-charging you for changes, just don't send them another request for proposal, but don't think everybody behaves like that.

Or maybe, you just bought under-priced software, and the company is getting back the money this way? Hmmm....

How to test Spring session scoped beans

noreply@blogger.com (Jakub Nabrdalik) — Fri, 15 Apr 2011 11:58:00 +0000

I wanted to use the http session just as a repository (database/files), to keep facebook access token for currently logged user. While I can manipulate session directly, another option is to declare the class as a session scoped bean in Spring. Something like this:

public class RepositoryOnHttpSession {
    private String facebookAccessToken;

    public FacebookTemplate getFacebookTemplate() {
        return new FacebookTemplate(facebookAccessToken);
    }

    public void setFacebookAccessToken(String facebookAccessToken) {
        this.facebookAccessToken = facebookAccessToken;
    }    
}

<bean id="repositoryOnHttpSession" class="pl.touk.storytelling.infrastructure.repositories.RepositoryOnHttpSession" scope="session">
    <aop:scoped-proxy/>
</bean>

<aop:scoped-proxy/> makes Spring IoC container create a cglib proxy, and inject that to other singleton type beans instead. All nice and cool, except integration tests (which get Spring IoC container to inject all the dependencies) are blowing up with:

java.lang.IllegalStateException: No Scope registered for scope 'session'

While there's a lot of solutions to be googled (including redeclaring the object as a prototype/sinlgeton for test context, injecting mock http session and request), the easiest way to have a simple thread-bound session scope is just to declare it in the TEST IoC configuration, like below. Just keep in mind that junit fires all tests in a single thread by default, so the state is persisted between tests. You may need to clean it up in @After.

<bean class="org.springframework.beans.factory.config.CustomScopeConfigurer">
    <property name="scopes">
        <map>
            <entry key="session">
                <bean class="org.springframework.context.support.SimpleThreadScope"/>
            </entry>
        </map>
    </property>
</bean>

Spring Security by example: OpenID (login via gmail)

noreply@blogger.com (Jakub Nabrdalik) — Fri, 01 Apr 2011 12:01:00 +0000

This is a part of a simple Spring Security tutorial:

1. Set up and form authentication
2. User in the backend (getting logged user, authentication, testing)
3. Securing web resources
4. Securing methods
5. OpenID (login via gmail)
6. OAuth2 (login via Facebook)
7. Writing on Facebook wall with Spring Social

OpenID is to form authentication, what DVCS is to centralized version control system.

Ok, but without the technical mumbo-jumbo: OpenID allows the user to use one account (like Gmail) to login to other services (websites) without having to remember anything and without worries about password security.

Easy enough?

And the best part is: chances are, you already have an account which is a provider to OpenID. Are you registered on Gmail, Yahoo, or even Blogger? You can already use it to login to other sites.

But why would you? Isn't login/password good enough?

Have you ever wondered whether the service you are logging to, uses one-way password hashing or keeps the password as open text? Most users do not create password per site because they cannot remember more than, let's say, three passwords. If one site keeps their password as open text, they are practically screwed. And more often than not, it's the site that has the worst (or non-existing) security. Welcome to hell: all your bases are belong to us.

By the way, I know of only one site, which does SHA hashing on the client side (in javascript), so that the server has no way of knowing what the real password is: sprezentuj.pl

Kudos for that, though it smells like overengineering a bit :)

OpenID is pretty simple, and the process description at Wikipedia is practically everything you need, but as they say, one picture is worth 1000 words, so let's look at a picture. There is a loot you can find with google, but most of them are in big-arrows-pointing-some-boxes-notation, and I find easier to read a sequence diagram, so here is my take on it:

Read emails from imap with Spring Intergration

noreply@blogger.com (Jakub Nabrdalik) — Fri, 01 Apr 2011 09:26:00 +0000

What's the easiest way to read emails from IMAP account in Java? Depends what your background is. If you have any experience in Apache Camel, ServiceMix, Mule, you already know the answer. If you don't, and your application is using Spring already, Spring-Integration may be the solution for you.

It's not a one-liner like what you could do with Camel, but it's still quite easy to understand.

Spring Integration has great docs and there are nice tutorials around, but if you just want to get it running first and dig into the docs later, here is a quick example for you.

To make it work, you need three steps:

How to get to Agile Warsaw (Aenima)

noreply@blogger.com (Jakub Nabrdalik) — Wed, 30 Mar 2011 16:37:00 +0000

Agile Warsaw is a great, open group, but there is one problem with where all the meetings take place: it's easy to get lost in the building if you have never been to Aenima before. Together with Piotr Modrzyk we've recorder a short video, that may help a bit all the newcomers.

Spring Security by example: securing methods

noreply@blogger.com (Jakub Nabrdalik) — Wed, 30 Mar 2011 15:52:00 +0000

Spring Security by example: securing web resources

noreply@blogger.com (Jakub Nabrdalik) — Wed, 30 Mar 2011 14:23:00 +0000

This is a part of a simple Spring Security tutorial:

1. Set up and form authentication
2. User in the backend (getting logged user, authentication, testing)
3. Securing web resources
4. Securing methods
5. OpenID (login via gmail)
6. OAuth2 (login via Facebook)
7. Writing on Facebook wall with Spring Social

Securing web resources means making sure that only users with granted authority will be able to visit given URL.

Let's say we have a page “protected.html” that should be accessed only by our admin. First thing we have to do, is to define which URLs are protected and which are accessible for anonymous user. We do that by adding intercept-url tags and a decision manager to the http tag:

Spring Security by example: user in the backend, testing

noreply@blogger.com (Jakub Nabrdalik) — Wed, 30 Mar 2011 13:02:00 +0000

This is a part of a simple Spring Security tutorial:

1. Set up and form authentication
2. User in the backend (getting logged user, authentication, testing)
3. Securing web resources
4. Securing methods
5. OpenID (login via gmail)
6. OAuth2 (login via Facebook)
7. Writing on Facebook wall with Spring Social

While login by form, OpenID, OAuth2 and so on may be cool, we may often need to do login/logout in the backend. Finally, we will need a way to get currently logged user. Working with Spring Security, we may do all of that using two classes: SecurityContextHolder and AuthenticationManager. Instead of explaining their API (docs are better) let me show you an implementation of an easy to use service. Our interface looks like this:

Spring Security by example: set up and form authentication

noreply@blogger.com (Jakub Nabrdalik) — Mon, 28 Mar 2011 16:36:00 +0000

Spring Security (former Acegi) is a Java library that handles authorization and authentication in web applications. Documentation on the project web site is, as expected from Spring Source, easy to read and use. I have a feeling though, that most of us first search Google for a fast, technology tutorial, before reading the docs, so in this little article I'm going to show you a few things Spring Security can do, give you a few hints and code snippets I have, after using it a little bit here and there. I'm not going to explain everything throughly, that's what docs are for, but what is here should help to get you started (or decide whether you want to).

Since this is quite a lot of text for a blog post.

Here is the plan:

1. Set up and form authentication
2. User in the backend (getting logged user, authentication, testing)
3. Securing web resources
4. Securing methods
5. OpenID (login via gmail)
6. OAuth2 (login via Facebook)
7. Writing on Facebook wall with Spring Social

Animating Developers, 4 months later

noreply@blogger.com (Jakub Nabrdalik) — Sun, 06 Feb 2011 18:46:00 +0000

Four months ago I had a chance to present Agile Skills Project and experiments we do at TouK, to create a learning ecosystem and culture of constant improvement. Time to report progress of our experiments.

Map of New Ideas

The request came from our HQ (Headquarters, which is our group of main company owners): lets use Jira to gather ideas and initiatives about how to improve our company. You know, Keizen style.

The reasoning is, that though some people, including myself, always loudly present their opinions, new concepts, and basically try to change the company from the ground up (whether the stakeholders like it or not), others need some encouragement and a safe way to suggest ideas for improvement.

Our map of new ideas is a simple Jira project, where anybody can suggest anything he/she thinks is worthwhile. The new idea is verified by a group responsible for deciding whether it's in line with company's profile and possible with our current resources. If so, the idea and needed resources, are assigned to someone who, from now on, is responsible for making it happen. It doesn't have to be the person proposing the idea, but quite often it is. The ideas are also grouped by the area they correspond to, one from culture, evolution, fitness, relations, survival or contribution.

As for the moment, our statistics look as follow:

Proposed: 36
Selected: 6 (waiting for a victim)
Assigned: 8 (but not started yet)
Ongoing: 6
Refused: 1
Finished: 26

Things that appear on the map vary, including for example other experiments described here (Weekly Workshops, OCRs), open sourcing some libraries, drawing diagrams of all the systems we create for our main clients (very useful for new developers) or creating database environment for regression tests on Oracle.

As you see, it goes quite well, and I believe it's a great way to get your developers involved in the company improvement. As a side effect, people feel they have more influence over where the company is heading, which is always good and gives a nice motivational kick (“Hey look, now I'm not only building software, I'm building my company as well!”).

Thing to remember: this is a map for company-wide ideas. In the beginning there was a confusion over whether stuff about how to improve one's project/team should be here, but since our teams are self-organized, there is no need to involve the rest of our company. The power is in one's hands anyway.

Weekly Workshops

Every Friday at 4pm, we have a one hour long technology workshop open to everyone. It usually takes the form of a lecture, with slides, some coding, and examples, but the form is open. Other features are not, and that is very important for it's success.

This is a concept I've borrowed from Supermedia Interactive, when Piotr Szarwas was still the head of the development department and my boss. I'm sure it's popular in many companies, but Piotr taught me how to do it correctly.

The goal is to share knowledge and learn new things.

The date and time are set in stone. Every Friday, 4pm. It's during our work hours, so we had to choose time, which won't stop or slow down our normal development. Since our sprints are week long, Friday 4pm is usually after the retrospective anyway. Let's face it: at 4pm on Friday, we are either busy putting down some fires, learning, or slacking off anyway.

Everyone can present anything, as long as there are people who want to listen to. We keep a calendar on confluence for coordination. We also have a page with suggested topics, where people put things they would like to listen about, for others to pick up and investigate. One rule is very important thought: if we do not have a volunteer for next Friday, we are going to pick one.

Yes, the participation (as a lecturer) is mandatory for everyone. It basically means, each developer should give at least one lecture per year. No excuses.

At first, there was some turmoil about that. People do not like to be forced to do anything. But the first drawing showed that it's a good idea. Randomly chosen people were giving great presentations. The reason is, that everyone has something interesting to show, it's just that a carrot of general appreciation is not enough sometimes. Sometimes you also need a stick. That's how a human mind works.

So what are our workshops about? Here are a few topic examples:

Clojure – lisp for java programmers.
Rapid Application Development using Grails and Vaadin
Smartclient: RAD even faster.
Apache Hadoop and projects around it.
JBoss Envers – easy entity auditing
How to get users from Gmail and Facebook: OAuth 2.0, OpenId, Spring Security and Spring Social in web applications

Weekly OCRs

I've already described that in here so I'm not going to repeat myself. What has changed, is that we have it on a weekly basis, we have it split in two: java and database OCRs separately, and that we have designated people responsible for making it happen (not for leading the meeting, but for choosing a victim, if there is no volunteer).

Java OCRs are held on Friday at 3pm, for all the same reasons as given above for workshops.

I've noticed that some people see it as a chance to brainstorm and refactor some really troublesome code, they work with, which they normally do not want or have no time to touch. That is a great idea, and since we use our own code for OCRs, and since we actually try to commit the changes, it's more like getting help for your project, than only sharing some thoughts.

Database teams do it in a little different way, but not being there, I'm not inclined to explain it.

Quest system during work hours

The quest ecosystem described on Agile Skills Project web site, about which I've been talking on Agile Warsaw is a great self-motivational tool, but the question we had, was how a company can help people get it started? My idea was, that having everyone choose his own mentor and booking an hour a month to talk with him about the progress or lack thereof, can do the trick. But that assumes, people are working back home to meet their goals, which is not true for everyone. Witek Wolejszo wanted to check whether making it a work-time activity will spread the technique to those, who don't want to use their free time.

Eight people with different background and free time activity were chosen for a month long experiment, where they would choose an educational goal and try to achieve it within working hours.

The feedback was generally negative. People complained for these main obstacles:

low quest priority (everything for the client is more important) ends with process starvation
quests as defined were things to be done alone, and we prefer working with other people (pairs)
no motivation from quest done for yourself only, when it doesn't have an immediate use and no one else is waiting for the outcomes

At the same time people didn't think it was a totally bad idea, but would rather like, if quests were somehow corresponding to our current products and projects and had defined timeboxes.

This made us close the experiment and start with two others, described below. We do not know yet how these will work out.

Task exchange board.

When creating a backlog for a project, there are usually some tasks for which the domain knowledge available to team members is not needed. This include investigation into new libraries and technologies (spikes), setting up things very technology oriented (auditing by Jboss Envers, excel export via Apache POI, etc.), creating black-box and/or open source libraries, writing proof of concepts or collecting and releasing some tools from the project as an Open Source libraries. All those tasks are candidates for Task Exchange Board.

When a backlog is done, the team can decide which tasks to put onto the board. These should be estimated like on a sprint planning, should have a defined time expectation or a timebox (in case of spikes) and Definition of Done.

Once on the Board, anyone in the company, even from different project or department, as long as he has enough time (i.e. his PM/team leader has nothing against it), can pick up an interesting task and do it using the budget of the team which created the task.

This way, people can do something important and interesting in a new technology, on full time, without changing the current project. If you have enough of what you are doing, this may be a productive brake for you. If you are currently only supporting some existing project or waiting for bug reports, why not to do something interesting and learn something new on the way? This may be a equivalent of a quest for those, who do not have any time at home.

These tasks, may also be done by members of the team which created them.

One important thing has to be understood: since the team creating the task doesn't know who is going to take it, it's up to the guy taking the task, to finish within or before estimated time. Basically, if you are not sure you can finish this on time, or do not want to risk spending your free hours on it, don't take it. This should not slow down the development time. If it did, no team would put another task on the board.

What technologies you want to work with?

This time-fridge contains
technologies we do not use anymore.
ZX-Spectrum, Commodore 64,
Commodore 128; Amstrad,
Macbook Air...
Old kind of stuff.
We are an old company :)

We have a team of guys responsible for assigning people to new projects or moving them between existing. This team has a sky level view on what's currently going on in the whole company, all new and planned projects.

So far, the team was making up decisions based on their own perspective of what one is capable of. We wanted to change it a bit, to make TouK a better place for people who feel a rush on some technology, so we created a confluence page, where everyone can submit what technologies he would like to work with.

Now, this looks very simple, but frankly speaking, I wasn't aware (and I believe nobody else was) of how often some of technologies would be mentioned by different people. As an effect, we can see now, that we have a lot of developers wanting to take part in a full-Grails, Java free project. While in theory, every team can decide for itself, what technologies it uses, seldom someone would take such a drastic, risky step, as changing the language for the whole project. By gathering information about what people would like to work with, we can create a team very dedicated and motivated to use some technology. That may pay off pretty well.

We shall see, whether the guys assigning people to projects, will take that into account.

More to go
There is a lot of stuff, we haven't tried yet. This includes for example mentoring, RPG character cards, exchanging knowledge with other companies by switching developers for a few days. Hopefully, I'll have more to report in a few months.

Hibernate hbm2ddl won't create schema before creating tables

noreply@blogger.com (Jakub Nabrdalik) — Tue, 18 Jan 2011 14:37:00 +0000

Situation

I have a local H2 in memory database for integration tests and an Oracle db for production. I do not control the Oracle DB model. The in memory H2 database is created automatically by adding

<prop key="hibernate.hbm2ddl.auto">update</prop>

to hibernate properties in AnnotationSessionFactoryBean. The definition of the entity stored in DB points to a schema

@Entity
@Table(name = "business_operations", schema = "sowa")
public class BusinessOperation {
...

The problem

When creating the H2 database, Hibernate won't create the schema before creating tables. As a result it will show errors when trying to create the tables in non existing schema and fail in any query (queries will be run with sowa.business_operations).

2011-01-18 15:13:30,884 INFO [org.hibernate.tool.hbm2ddl.SchemaUpdate] - Running hbm2ddl schema update
2011-01-18 15:13:30,885 INFO [org.hibernate.tool.hbm2ddl.SchemaUpdate] - fetching database metadata
2011-01-18 15:13:30,915 INFO [org.hibernate.tool.hbm2ddl.SchemaUpdate] - updating schema
2011-01-18 15:13:30,927 INFO [org.hibernate.tool.hbm2ddl.DatabaseMetadata] - table not found: business_operations
2011-01-18 15:13:30,941 ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - Unsuccessful: create table sowa.business_operations 
2011-01-18 15:13:30,942 ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - 
Schema "SOWA" not found;

Turns out this bug is reported and open since 2006: link.

The solution

The solution to this problem is to create the schema before hibernate's hbm2ddl turns on. That would be easy with H2 if we could tell H2 to initialize itself like this:

database.url=jdbc:h2:mem:;INIT=RUNSCRIPT FROM 'src/main/resources/scripts/create.sql';

All seems nice, except H2 RUNSCRIPT FROM command doesn't work with relative resources as you may expect. Fortunatelly INIT allows us to give any commands, not just point to a script, so this little change will solve the problem:

database.url=jdbc:h2:mem:;INIT=create schema IF NOT EXISTS sowa

Yeah, I know it's obvious and simple stupid, but looking at all the questions on all the mailing lists in google I may have just saved a little bit of somebody's time.

Google Guava v07 examples

noreply@blogger.com (Jakub Nabrdalik) — Sun, 31 Oct 2010 12:31:00 +0000

We have something called Weekly Technology Workshops at TouK, that is, every Friday at 16:00 somebody has a presentation for everyone willing to come. We present stuff we learn and work on at home, but we also have a bulletin board with topics that people would like to listen about. Last week Maciej Próchniak had a talk about Clojure, this time a few folks asked for an introduction to Google Guava libraries.

Since this was a dead simple task, I was happy to deliver.

WTF is Guava?

It's a set of very simple, basic classes, that you end up writing yourself anyway. Think in terms of Apache commons, just by Google. Just to make your life a little bit easier.

There is an early (v04) presentation and there was a different one (in Polish) at Javarsowia 2010 by Wiktor Gworek.

At the time of writing this, the latest version is v07, it's been mavenized and is available at a public maven repo.

Here's a quick review of a few interesting things. Don't expect anything fancy though, Guava is very BASIC.

@VisibleForTesting

A simple annotation that tells you why a particular property access restriction has been relaxed.

A common trick to use in testing is to relax access restrictions to default for a particular property, so that you can use it in a unit test, which resides in the same package (though in different catalog). Whether you thing it's good or bad, remember to give a hint about that to the developer.

Consider:

public class User {
    private Long id;
    private String firstName;
    private String lastName;
    String login;

Why is login package scoped?

public class User {
    private Long id;
    private String firstName;
    private String lastName;
    @VisibleForTesting String login;

Ah, that's why.

Preconditions

Guava has a few preconditions for defensive programming (Design By Contract), but they are not quite as good as what Apache Commons / Spring framework has. One thing interesting is that Guava solution returns the object, so could be inlined. Consider:

Using hand written preconditions:

public User(Long id, String firstName, String lastName, String login) {
        validateParameters(id, firstName, lastName, login);
        this.id = id;
        this.firstName = firstName;
        this.lastName = lastName;
        this.login = login.toLowerCase();
    }

    private void validateParameters(Long id, String firstName, String lastName, String login) {
        if(id == null ) {
            throw new IllegalArgumentException("id cannot be null");
        }

        if(firstName == null || firstName.length() == 0) {
            throw new IllegalArgumentException("firstName cannot be empty");
        }

        if(lastName == null || lastName.length() == 0) {
            throw new IllegalArgumentException("lastName cannot be empty");
        }

        if(login == null || login.length() == 0) {
            throw new IllegalArgumentException("login cannot be empty");
        }
    }

Using guava preconditions:

public void fullyImplementedGuavaConstructorWouldBe(Long id, String firstName, String lastName, String login) {
        this.id = checkNotNull(id);
        this.firstName = checkNotNull(firstName);
        this.lastName = checkNotNull(lastName);
        this.login = checkNotNull(login);

        checkArgument(firstName.length() > 0);
        checkArgument(lastName.length() > 0);
        checkArgument(login.length() > 0);
    }

(Thanks Yom for noticing that checkNotNull must go before checkArgument, though it makes it a bit unintuitive)

Using spring or apache commons preconditions (the use looks exactly the same for both libraries):

public void springConstructorWouldBe(Long id, String firstName, String lastName, String login) {
        notNull(id); hasText(firstName); hasText(lastName); hasText(login);
        this.id = id;
        this.firstName = firstName;
        this.lastName = lastName;
        this.login = login;
    }

CharMatcher

For people who hate regexp or just want a simple and good looking object style pattern matching solution.

Examples:

And/or ease of use

        String input = "This invoice has an id of 192/10/10";
        CharMatcher charMatcher = CharMatcher.DIGIT.or(CharMatcher.is('/'));
        String output = charMatcher.retainFrom(input);

output is: 192/10/10

Negation:

        String input = "DO NOT scream at me!";
        CharMatcher charMatcher = CharMatcher.JAVA_LOWER_CASE.or(CharMatcher.WHITESPACE).negate();
        String output = charMatcher.retainFrom(input);

output is: DONOT!

Ranges:

        String input = "DO NOT scream at me!";
        CharMatcher charMatcher = CharMatcher.inRange('m', 's').or(CharMatcher.is('a').or(CharMatcher.WHITESPACE));
        String output = charMatcher.retainFrom(input);

output is: sram a m

Joiner / Splitter

As the names suggest, it's string joining/splitting done the right way, although I find the inversion of calls a bit... oh well, it's java.

        String[] fantasyGenres = {"Space Opera", "Horror", "Magic realism", "Religion"};
        String joined = Joiner.on(", ").join(fantasyGenres);

Output: Space Opera, Horror, Magic realism, Religion

You can skip nulls:

        String[] fantasyGenres = {"Space Opera", null, "Horror", "Magic realism", null, "Religion"};
        String joined = Joiner.on(", ").skipNulls().join(fantasyGenres);

Output: Space Opera, Horror, Magic realism, Religion

You can fill nulls:

        String[] fantasyGenres = {"Space Opera", null, "Horror", "Magic realism", null, "Religion"};
        String joined = Joiner.on(", ").useForNull("NULL!!!").join(fantasyGenres);

Output: Space Opera, NULL!!!, Horror, Magic realism, NULL!!!, Religion

You can join maps

        Map<Integer, String> map = newHashMap();
        map.put(1, "Space Opera");
        map.put(2, "Horror");
        map.put(3, "Magic realism");
        String joined = Joiner.on(", ").withKeyValueSeparator(" -> ").join(map);

Output: 1 → Space Opera, 2 → Horror, 3 → Magic realism

Split returns Iterable instead of JDK arrays:

        String input = "Some very stupid data with ids of invoces like 121432, 3436534 and 8989898 inside";
        Iterable<String> splitted = Splitter.on(" ").split(input);

Split does fixed length splitting, although you cannot give a different length for each “column” which makes it's use a bit limited while parsing some badly exported excels.

        String input =
                "A  1  1  1  1\n" +
                "B  1  2  2  2\n" +
                "C  1  2  3  3\n" +
                "D  1  2  5  3\n" +
                "E  3  2  5  4\n" +
                "F  3  3  7  5\n" +
                "G  3  3  7  5\n" +
                "H  3  3  9  7";
        Iterable<String> splitted = Splitter.fixedLength(3).trimResults().split(input);

You can use CharMatcher while splitting

        String input = "Some very stupid data with ids of invoces like 123231/fv/10/2010, 123231/fv/10/2010 and 123231/fv/10/2010";
        Iterable<String> splitted = Splitter.on(CharMatcher.DIGIT.negate())
                                            .trimResults()
                                            .omitEmptyStrings()
                                            .split(input);

Predicates / Functions

Predicates alone are not much, it's just an interface with a method that returns true, but if you combine predicates with functions and Collections2 (a guava class that simplifies working on collections), you get a nice tool in your toolbox.

But let's start with basic predicate use. Imagine we want to find whether there are users who have logins with digits inside. The inocation would be (returns boolean):

Predicates.in(users).apply(shouldNotHaveDigitsInLoginPredicate);

And the predicate looks like that

public class ShouldNotHaveDigitsInLoginPredicate implements Predicate<User> {
    @Override
    public boolean apply(User user) {
        checkNotNull(user);
        return CharMatcher.DIGIT.retainFrom(user.login).length() == 0;
    }    
}

Now lets add a function that will transform a user to his full name:

public class FullNameFunction implements Function<User, String> {
    @Override
    public String apply(User user) {
        checkNotNull(user);
        return user.getFirstName() + " " + user.getLastName();
    }    
}

You can invoke it using static method transform:

List<User> users = newArrayList(new User(1L, "sylwek", "stall", "rambo"),
  new User(2L, "arnold", "schwartz", "commando"));

List<String> fullNames = transform(users, new FullNameFunction());

And now lets combine predicates with functions to print names of users that have logins which do not contain digits:

List<User> users = newArrayList(new User(1L, "sylwek", "stall", "rambo"), 
  new User(2L, "arnold", "schwartz", "commando"), 
  new User(3L, "hans", "kloss", "jw23"));

Collection<User> usersWithoutDigitsInLogin = filter(users, new ShouldNotHaveDigitsInLoginPredicate());
String names = Joiner.on("\n").join( transform(usersWithoutDigitsInLogin, new FullNameFunction()) );

What we do not get: fold (reduce) and tuples. Oh well, you'd probably turn to Java Functional Library anyway, if you wanted functions in Java, right?

CaseFormat

Ever wanted to turn those ugly PHP Pear names into nice java/cpp style with one liner? No? Well, anyway, you can:

String pearPhpName = "Really_Fucked_Up_PHP_PearConvention_That_Looks_UGLY_because_of_no_NAMESPACES";
String javaAndCPPName = CaseFormat.UPPER_UNDERSCORE.to(CaseFormat.UPPER_CAMEL , pearPhpName);

Output: ReallyFuckedUpPhpPearconventionThatLooksUglyBecauseOfNoNamespaces

But since Oracle has taken over Sun, you may actually want to turn those into sql style, right?

        String sqlName = CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_UNDERSCORE, javaAndCPPName);

Output: really_fucked_up_php_pearconvention_that_looks_ugly_because_of_no_namespaces

Collections

Guava has a superset of Google collections library 1.0, and this indeed is a very good reason to include this dependency in your poms. I won't even try to describe all the features, but just to point out a few nice things:

you have an Immutable version of pretty much everything
you get a few nice static and statically typed methods on common types like Lists, Sets, Maps, ObjectArrays, which include:

easy way of creating based on return type: e.g. newArrayList
transform (way to apply functions that returns Immutable version)
partition (paging)
reverse

And now for a few more interesting collections.

Mutlimaps

Mutlimap is basically a map that can have many values for a single key. Ever had to create a Map<T1, Set<T2>> in your code? You don't have to anymore.

Multimap<Integer, String> multimap = HashMultimap.create();
        multimap.put(1, "a");
        multimap.put(2, "b");
        multimap.put(3, "c");
        multimap.put(1, "a2");

There are of course immutable implementations as well: ImmutableListMultimap, ImmutableSetMultomap, etc.

You can construct immutables either in line (up to 5 elements) or using a builder:

Multimap<Integer, String> multimap = ImmutableSetMultimap.of(1, "a", 2, "b", 3, "c", 1, "a2"); 
Multimap<Integer, String> multimap = new ImmutableSetMultimap.Builder<Integer, String>()
        .put(1, "a")
        .put(2, "b")
        .put(3, "c")
        .put(1, "a2")
        .build();

BiMap

BiMap is a map that have only unique values. Consider this:

@Test(expected = IllegalArgumentException.class)
public void biMapShouldOnlyHaveUniqueValues() {
 BiMap<Integer, String> biMap = HashBiMap.create();
 biMap.put(1, "a");
 biMap.put(2, "b");
 biMap.put(3, "a"); //argh! an exception
}

That allows you to inverse the map, so the values become key and the other way around:

BiMap<Integer, String> biMap = HashBiMap.create();
biMap.put(1, "a");
biMap.put(2, "b");
biMap.put(3, "c");

BiMap<String, Integer> invertedMap = biMap.inverse();

Not sure what I'd actually want to use it for.

Constraints

This allows you to add constraint checking on a collection, so that only values which pass the constraint may be added.

Imagine we want a collections of users with first letter 'r' in their logins.

Constraint<User> loginMustStartWithR = new Constraint<User>() {
    @Override
    public User checkElement(User user) {
        checkNotNull(user);
        
        if(!user.login.startsWith("r")) {
            throw new IllegalArgumentException("GTFO, you are not Rrrrrrrrr");
        }

        return user;
    }
};

And now for a test:

@Test(expected = IllegalArgumentException.class)
public void shouldConstraintCollection() {
 //given
 Collection<User> users = newArrayList(new User(1L, "john", "rambo", "rambo"));
 Collection<User> usersThatStartWithR = constrainedCollection(users, loginMustStartWithR);

 //when
 usersThatStartWithR.add(new User(2L, "arnold", "schwarz", "commando"));
}

You also get notNull constraint out of the box:

//notice it's not an IllegalArgumentException :( 
@Test(expected = NullPointerException.class)
public void notNullConstraintShouldWork() {
 //given
 Collection<Integer> users = newArrayList(1);
 Collection<Integer> notNullCollection = constrainedCollection(users, notNull());

 //when
 notNullCollection.add(null);
}

Thing to remember: constraints are not checking the data already present in a collection.

Tables

Just as expected, a table is a collection with columns, rows and values. No more Map<T1, Map<T2, T3>> I guess. The usage is simple and you can transpose:

Table<Integer, String, String> table = HashBasedTable.create();
table.put(1, "a", "1a");
table.put(1, "b", "1b");
table.put(2, "a", "2a");
table.put(2, "b", "2b");

Table transponedTable = Tables.transpose(table);

That's all, folks. I didn't present util.concurent, primitives, io and net packages, but you probably already know what to expect.

Java Developers' Day 2010 review

noreply@blogger.com (Jakub Nabrdalik) — Sun, 17 Oct 2010 13:46:00 +0000

On the 7-8th of October 2010, Cracow held Java Developers' Day conference. This year it was two days long, so I guess they'll have to think about changing the name. My expectations weren't very high. First of all, I've heard an opinion that JDD is getting worse every year. Second, for the same price as GeeCON you got only one track. Third some of the lectures seemed really uninspiring.

For example, I was afraid, that the one about Flex is going to be the same one I've seen on 4Developers, half a year ago in Poznań. And, of course, there was the one most controversial to all the people I had a chance speaking with, the sponsored lecture from Wipro Technologies, titled: “Wipro in Europe and development opportunities on Polish market”.

Doesn't sound like something you'd like to listen to on a Java conference, does it? More like an advertisement, to me.

Fortunately, my doubts were mostly unfounded.

The first day started a little earlier than planned, with Bill Burke talk about RESTful Java. Quite nice, I must say, as long as you have no idea what REST means, as half of the lecture was very basic. The other part was about JAX-RS (and RESTEasy implementation), and that's where it got my attention. I haven't had a chance to use JAX-RS yet, but the simplicity and efficiency of it is very appealing. I'll have to give it a shot, especially when .NET/Java web service integration is sometimes very painful.

The second talk, “Java programming in days of multi-core processors”, by Angelika Langer, was gorgeous. Maybe it's because of my limited experience in concurrency, maybe because Angelika presented a more in-depth view on the things happening under the hood, than I am used to, and maybe because she threw away the incorrect model most books present. What I'm sure about, is that Angelika is a great trainer and speaker, with vast knowledge and expertise. It was a pure pleasure to listen to her and I can only hope to be so passionate and sharp at her age.

Not that she's that old, mind you. It's just that terms like “burned-out” and “tired” do not seem to have anything to do with her.

Then there was Jarosław Błąd from e-point, talking about performance tests in JEE. That was also quite nice, though a little too basic. I wish the speaker could show us a little more real case scenarios and stories, as it was obvious he had a lot of interesting thoughts on that matter, but probably because of NDAs, he decided to go with more theoretical and generic information instead. Anyway, this was a sponsored talk done right. Thanks e-point for not leaving us with just advertisements.

After the lunch came the hit of the day, Ted Neward talk about functional programming for busy developer. A few slides passed by, when Ted asked the audience, whether we would rather see presentation or life coding. Guess what the answer was.

The great thing about the lecture was that Ted didn't use anything more than standard Java, to show us the benefits of thinking in terms of functional languages. The examples were practical, with stuff you can really find from time to time in your code, and the advantages clear and explicit. Somewhere in the middle of the show, Ted said, that he wants us to remember, that we do not have to use anything fancy like Scala, to start solving some classes of problems in a much better way. I only wish he had more time on his hands, but I was lucky to sign in for Scala workshops with him on Friday.

I didn't go for Flex presentation, partially because of the beforementioned doubts, partially because I've met some friends and speakers on the way. I really wish I could be there, on their lectures, especially on Łukasz Kuczera talk about Lift+Comet and Łukasz Szydło presentation about Apprenticeship, but I could either do that or go for the workshop with Ted Neward, and after what Ted had shown us a few hours before, I was sure his workshop will be a mind opener.

And here is for all those anxious about just one track on JDD10. There were actually two on Friday, if you count the workshops, and even though that doesn't seem like much, the quality of what Ted had to offer, beat up the disadvantage of not being able to change every session for something different.

The last lecture on Thursday was “Brave changes: how to make new ideas happen”, given by Linda Rising. While not Java specific, that was quite interesting to me, mostly because of the latest changes I've been part of at TouK (both my own initiatives that you can read about here and here, and overall works on defining company goals and vision). Thing to remember: what your audience is eating is more important than what they are listening to. Scary but true.

Then there was the integration party. And as expected from programmers, Nintendo Wii had a much bigger take than girls :)

For three hours on Friday, I've been enjoying Ted Neward's Scala workshops. I won't give you much details, except it was really great, since Witek Wołejszo wrote a nice summary already.

And I didn't dare to go for “Wipro in Europe and development opportunities on Polish market”. I was afraid, that my positive experience from JDD10 could be a bit reduced.

Overall, another great conference. Thanks to Witek Wołejszo, Piotr Przybyłek and Tomasz Dziurko for this interesting trip.

Wicket form submit not safe for redirecting to intercept page

noreply@blogger.com (Jakub Nabrdalik) — Wed, 13 Oct 2010 17:00:00 +0000

The problem

When you have a form, that anybody can see, but only logged on users can POST, you may want to redirect the user to the login page, and back to the form after login

Using wicket 1.3/1.4, if you do that using redirectToInterceptPage(loginPage) or RestartResponseAtInterceptPageException, after returning, the client will loose all the data entered to the form.

The details

The reason why this happens, is because of how redirectToInterceptPage works. It saves the URL of the requested page, and later, when continueToOriginalDestination is called, it redirects the client to the saved URL using GET. When the last call from the client was a non-ajax POST to the form, the client will be redirected without any posted data. Wicket will handle the situation issuing HTTP 302 and redirecting the user again, but all the data is already lost.

The funny thing is that the data is actually getting to the form, because of the first POST, but then it's overwritten with nulls on the redirected GET. To make it clear, here's the HTTP conversation:

Client: POST http://localhost:8080/test?wicket:interface=:3:form::IFormSubmitListener:: (post to the form)
Server: HTTP 302 Moved Temporarily (the input was parsed, the model was updated, but you are being redirected to the login page because of redirectToInterceptPage or exception)
Client: GET http://localhost:8080/?wicket:interface=:4:::: 
Server: HTTP 200 OK (server is responding with the login page)
Client: POST  https://localhost:8443/j_spring_security_check.... (post login and password, here using spring security)
Server: HTTP 302 Moved Temporarily (validation is done. Now you are redirected from spring security to the page with wicket redirectToInterceptPage)
Client: GET https://localhost:8443/redirectAfterLogin  (here  redirectToInterceptPage is called)
Server: HTTP 302 Moved Temporarily (you are being redirected the original URL)
Client: GET http://localhost:8080/test?wicket:interface=:3:form::IFormSubmitListener:: (the same URL as the first POST but this time without post data. now your form is being submitted again, but with nulls instead of entered data)
Server: HTTP 302 Moved Temporarily (being redirected by wicket, because of Redirect After Post pattern)
Client: GET http://localhost:8080/?wicket:interface=:3:1::: (back on the form page)
Server: HTTP 200 OK (the form is empty by now)

As you see, if wicket would not redirect you at the end to the url requested by POST, but to the one called by last GET before the original POST, your data would be there.

The issue was reported two years ago. Doesn't look like it's getting fixed any time soon.

The walkaround

If you can require your users to be logged in before you show them the form, you are safe. If not, you can submit the form by AJAX. This will solve the problem, because wicket will recognize, that it cannot redirect the user to the AJAX POST target (<ajax-response> is not exactly what the user would like to have rendered in the browser), and will redirect with GET to the URL of the last page instead, which was also requested by GET. And since the data was converted to the form model in the first POST (line 1), all is well.

And in case you don't want to have a partial page update via AJAX, but would rather like to render the whole page, all you need to do is add setResponsePage(getPage()) to your button. For example like this:

    class AjaxSendButton extends AjaxFallbackButton {
        public AjaxSendButton(String id, Form form) {
            super(id, form);
        }

        @Override
        protected void onSubmit(AjaxRequestTarget target, Form form) {
            //process your form input here
            setResponsePage(getPage());
        }
    }

Now your ajax form behaves just like a non ajax form, but can be redirected to an intercept page

The catch

When submitting forms via AJAX you have to be aware, that your form may be submitted without your submit button being clicked on. This may have unforseen consequences. For the whole problem description and a solution go here

Video from my presentation at Agile Warsaw

noreply@blogger.com (Jakub Nabrdalik) — Tue, 05 Oct 2010 19:34:00 +0000

Here's the video from my presentation and the discussion about Agile Skills Project and our experiments in motivating developers at my company, that I had a chance to show at Agile Warsaw.

Do not ask my why the camera is 100% time focused on the wall, I have no freaking idea :) The voices are there, and that matters.

You can either watch it on Parleys: http://parleys.com/d/2019 or right here. Be warned: it's in Polish.