Read emails from imap with Spring Intergration

What's the easiest way to read emails from IMAP account in Java? Depends what your background is. If you have any experience in Apache Camel, ServiceMix, Mule, you already know the answer. If you don't, and your application is using Spring already, Spring-Integration may be the solution for you.

It's not a one-liner like what you could do with Camel, but it's still quite easy to understand.

Spring Integration has great docs and there are nice tutorials around, but if you just want to get it running first and dig into the docs later, here is a quick example for you.

To make it work, you need three steps:

How to get to Agile Warsaw (Aenima)

Agile Warsaw is a great, open group, but there is one problem with where all the meetings take place: it's easy to get lost in the building if you have never been to Aenima before. Together with Piotr Modrzyk we've recorder a short video, that may help a bit all the newcomers.

Spring Security by example: securing methods

This is a part of a simple Spring Security tutorial:

1. Set up and form authentication
2. User in the backend (getting logged user, authentication, testing)
3. Securing web resources
4. Securing methods
5. OpenID (login via gmail)
6. OAuth2 (login via Facebook)
7. Writing on Facebook wall with Spring Social

Securing web resources is all nice and cool, but in a well designed application it's more natural to secure methods (for example on backend facade or even domain objects). While we may get away with role-based authorization in many intranet business applications, nobody will ever handle assigning roles to users in a public, free to use Internet service. We need authorization based on rules described in our domain.

For example: there is a service AlterStory, that allows cooperative writing of stories, where one user is a director (like a movie director), deciding which chapter proposed by other authors should make it to the final story.

The method for accepting chapters, looks like this:

Spring Security by example: securing web resources

This is a part of a simple Spring Security tutorial:

1. Set up and form authentication
2. User in the backend (getting logged user, authentication, testing)
3. Securing web resources
4. Securing methods
5. OpenID (login via gmail)
6. OAuth2 (login via Facebook)
7. Writing on Facebook wall with Spring Social

Securing web resources means making sure that only users with granted authority will be able to visit given URL.

Let's say we have a page “protected.html” that should be accessed only by our admin. First thing we have to do, is to define which URLs are protected and which are accessible for anonymous user. We do that by adding intercept-url tags and a decision manager to the http tag:

Spring Security by example: user in the backend, testing

This is a part of a simple Spring Security tutorial:

1. Set up and form authentication
2. User in the backend (getting logged user, authentication, testing)
3. Securing web resources
4. Securing methods
5. OpenID (login via gmail)
6. OAuth2 (login via Facebook)
7. Writing on Facebook wall with Spring Social

While login by form, OpenID, OAuth2 and so on may be cool, we may often need to do login/logout in the backend. Finally, we will need a way to get currently logged user. Working with Spring Security, we may do all of that using two classes: SecurityContextHolder and AuthenticationManager. Instead of explaining their API (docs are better) let me show you an implementation of an easy to use service. Our interface looks like this:

Spring Security by example: set up and form authentication

Spring Security (former Acegi) is a Java library that handles authorization and authentication in web applications. Documentation on the project web site is, as expected from Spring Source, easy to read and use. I have a feeling though, that most of us first search Google for a fast, technology tutorial, before reading the docs, so in this little article I'm going to show you a few things Spring Security can do, give you a few hints and code snippets I have, after using it a little bit here and there. I'm not going to explain everything throughly, that's what docs are for, but what is here should help to get you started (or decide whether you want to).

Since this is quite a lot of text for a blog post.

Here is the plan:

1. Set up and form authentication
2. User in the backend (getting logged user, authentication, testing)
3. Securing web resources
4. Securing methods
5. OpenID (login via gmail)
6. OAuth2 (login via Facebook)
7. Writing on Facebook wall with Spring Social

Animating Developers, 4 months later

Four months ago I had a chance to present Agile Skills Project and experiments we do at TouK, to create a learning ecosystem and culture of constant improvement. Time to report progress of our experiments.

Map of New Ideas

The request came from our HQ (Headquarters, which is our group of main company owners): lets use Jira to gather ideas and initiatives about how to improve our company. You know, Keizen style.

The reasoning is, that though some people, including myself, always loudly present their opinions, new concepts, and basically try to change the company from the ground up (whether the stakeholders like it or not), others need some encouragement and a safe way to suggest ideas for improvement.

Our map of new ideas is a simple Jira project, where anybody can suggest anything he/she thinks is worthwhile. The new idea is verified by a group responsible for deciding whether it's in line with company's profile and possible with our current resources. If so, the idea and needed resources, are assigned to someone who, from now on, is responsible for making it happen. It doesn't have to be the person proposing the idea, but quite often it is. The ideas are also grouped by the area they correspond to, one from culture, evolution, fitness, relations, survival or contribution.

As for the moment, our statistics look as follow:

• Proposed: 36
• Selected: 6 (waiting for a victim)
• Assigned: 8 (but not started yet)
• Ongoing: 6
• Refused: 1
• Finished: 26

Things that appear on the map vary, including for example other experiments described here (Weekly Workshops, OCRs), open sourcing some libraries, drawing diagrams of all the systems we create for our main clients (very useful for new developers) or creating database environment for regression tests on Oracle.

As you see, it goes quite well, and I believe it's a great way to get your developers involved in the company improvement. As a side effect, people feel they have more influence over where the company is heading, which is always good and gives a nice motivational kick (“Hey look, now I'm not only building software, I'm building my company as well!”).

Thing to remember: this is a map for company-wide ideas. In the beginning there was a confusion over whether stuff about how to improve one's project/team should be here, but since our teams are self-organized, there is no need to involve the rest of our company. The power is in one's hands anyway.


Weekly Workshops

Every Friday at 4pm, we have a one hour long technology workshop open to everyone. It usually takes the form of a lecture, with slides, some coding, and examples, but the form is open. Other features are not, and that is very important for it's success.

This is a concept I've borrowed from Supermedia Interactive, when Piotr Szarwas was still the head of the development department and my boss. I'm sure it's popular in many companies, but Piotr taught me how to do it correctly.

The goal is to share knowledge and learn new things.

The date and time are set in stone. Every Friday, 4pm. It's during our work hours, so we had to choose time, which won't stop or slow down our normal development. Since our sprints are week long, Friday 4pm is usually after the retrospective anyway. Let's face it: at 4pm on Friday, we are either busy putting down some fires, learning, or slacking off anyway.

Everyone can present anything, as long as there are people who want to listen to. We keep a calendar on confluence for coordination. We also have a page with suggested topics, where people put things they would like to listen about, for others to pick up and investigate. One rule is very important thought: if we do not have a volunteer for next Friday, we are going to pick one.

Yes, the participation (as a lecturer) is mandatory for everyone. It basically means, each developer should give at least one lecture per year. No excuses.

At first, there was some turmoil about that. People do not like to be forced to do anything. But the first drawing showed that it's a good idea. Randomly chosen people were giving great presentations. The reason is, that everyone has something interesting to show, it's just that a carrot of general appreciation is not enough sometimes. Sometimes you also need a stick. That's how a human mind works.

So what are our workshops about? Here are a few topic examples:

• Clojure – lisp for java programmers.
• Rapid Application  Development using Grails and Vaadin
• Smartclient: RAD even faster.
• Apache Hadoop and projects around it.
• JBoss Envers – easy entity auditing
• How to get users from Gmail and Facebook: OAuth 2.0, OpenId, Spring Security and Spring Social in web applications

Weekly OCRs

I've already described that  in here so I'm not going to repeat myself. What has changed, is that we have it on a weekly basis, we have it split in two: java and database OCRs separately, and that we have designated people responsible for making it happen (not for leading the meeting, but for choosing a victim, if there is no volunteer).

Java OCRs are held on Friday at 3pm, for all the same reasons as given above for workshops.

I've noticed that some people see it as a chance to brainstorm and refactor some really troublesome code, they work with, which they normally do not want or have no time to touch. That is a great idea, and since we use our own code for OCRs, and since we actually try to commit the changes, it's more like getting help for your project, than only sharing some thoughts.

Database teams do it in a little different way, but not being there, I'm not inclined to explain it.


Quest system during work hours

The quest ecosystem described on Agile Skills Project web site, about which I've been talking on Agile Warsaw is a great self-motivational tool, but the question we had, was how a company can help people get it started? My idea was, that having everyone choose his own mentor and booking an hour a month to talk with him about the progress or lack thereof, can do the trick. But that assumes, people are working back home to meet their goals, which is not true for everyone. Witek Wolejszo wanted to check whether making it a work-time activity will spread the technique to those, who don't want to use their free time.

Eight people with different background and free time activity were chosen for a month long experiment, where they would choose an educational goal and try to achieve it within working hours.

The feedback was generally negative. People complained for these main obstacles:

low quest priority (everything for the client is more important) ends with process starvation
quests as defined were things to be done alone, and we prefer working with other people (pairs)
no motivation from quest done for yourself only, when it doesn't have an immediate use and no one else is waiting for the outcomes

At the same time people didn't think it was a totally bad idea, but would rather like, if quests were somehow corresponding to our current products and projects and had defined timeboxes.

This made us close the experiment and start with two others, described below. We do not know yet how these will work out.


Task exchange board.

When creating a backlog for a project, there are usually some tasks for which the domain knowledge available to team members is not needed. This include investigation into new libraries and technologies (spikes), setting up things very technology oriented (auditing by Jboss Envers, excel export via  Apache POI, etc.), creating black-box and/or open source libraries, writing proof of concepts or collecting and releasing some tools from the project as an Open Source libraries. All those tasks are candidates for Task Exchange Board.

When a backlog is done, the team can decide which tasks to put onto the board. These should be estimated like on a sprint planning, should have a defined time expectation or a timebox (in case of spikes) and Definition of Done.

Once on the Board, anyone in the company, even from different project or department, as long as he has enough time (i.e. his PM/team leader has nothing against it), can pick up an interesting task and do it using the budget of the team which created the task.

This way, people can do something important and interesting in a new technology, on full time, without changing the current project. If you have enough of what you are doing, this may be a productive brake for you. If you are currently only supporting some existing project or waiting for bug reports, why not to do something interesting and learn something new on the way? This may be a equivalent  of a quest for those, who do not have any time at home.

These tasks, may also be done by members of the team which created them.

One important thing has to be understood: since the team creating the task doesn't know who is going to take it, it's up to the guy taking the task, to finish within or before estimated time. Basically, if you are not sure you can finish this on time, or do not want to risk spending your free hours on it, don't take it. This should not slow down the development time. If it did, no team would put another task on the board.


What technologies you want to work with?

This time-fridge contains
technologies we do not use anymore.
ZX-Spectrum, Commodore 64,
Commodore 128; Amstrad,
Macbook Air...
Old kind of stuff.
We are an old company :)

We have a team of guys responsible for assigning people to new projects or moving them between existing. This team has a sky level view on what's currently going on in the whole company, all new and planned projects.

So far, the team was making up decisions based on their own perspective of what one is capable of. We wanted to change it a bit, to make TouK a better place for people who feel a rush on some technology, so we created a confluence page, where everyone can submit what technologies he would like to work with.

Now, this looks very simple, but frankly speaking, I wasn't aware (and I believe nobody else was) of how often some of technologies would be mentioned by different people. As an effect, we can see now, that we have a lot of developers wanting to take part in a full-Grails, Java free project. While in theory, every team can decide for itself, what technologies it uses, seldom someone would take such a drastic, risky step, as changing the language for the whole project. By gathering information about what people would like to work with, we can create a team very dedicated and motivated to use some technology. That may pay off pretty well.

We shall see, whether the guys assigning people to projects, will take that into account.

More to go
There is a lot of stuff, we haven't tried yet. This includes for example mentoring, RPG character cards, exchanging knowledge with other companies by switching developers for a few days. Hopefully, I'll have more to report in a few months.