Report of software engineering

University Degree Mathematical and Computer Sciences

Large software development project that used in Extreme Programming

Junwang

Abstract

The agile software development process called Extreme Programming (XP) is a set of best practices which, when used, promises swifter delivery of quality software than one finds with more traditional methodologies. At ThoughtWorks they have adopted a modified version of XP that has been tailored through their experiences to suit a large project. It was included many aspects such as XP on a large project(ATLAS), recognizing and responding to “Bad Smells” in extreme programming, Extreme Programming and Database Administration, one of the most time-consuming and unfulfilling activities in testing is coding and maintaining test objects(ObjectMother Pattern).

1. Introduction

Thoughtworks, Inc. is a Chicago-based system integration and consulting company of around 300 people. They specialize in building demanding business applications using technology that is both new and strategic for the industry. They also work on complex business applications that usually require stitching together various components and augmenting these components with complicated and changing business logic. They’ve done a lot of work in the leasing industry, which is not known for simple and logical contractual agreements.

The particular examples here will explore are J2EE development project, Database Administrator (DBA) or DBA team, ATLAS, ObjectMother Pattern, which switched to an extreme programming (XP) process after a more traditional approach proved ineffective. These projects consisted of a 50-person team, about 30 of which were developers, working over a three year period. The application being built was a comprehensive enterprise resource solution for the leasing industry, including all aspects of accounts receivable, asset management, and contract terminations. Especially instance is J2EE project, the present code base consists of well over 500,000 lines of executable code, with very little of it remaining from the earlier development effort using a different process. They have been members of the project team for over two years, and the examples present in this report were directly experienced in the day-to-day tasks performed during that time. Although XP proved to be a more effective and successful software development process on this project, which we learned and wish to show on this report is susceptible to making “wrong turns” that could prove to thwart a team’s ability to deliver quality products in a timely manner.

2. Recognizing and Responding to “Bad Smells” in Extreme Programming

In this part, it describes a large software development project that used a modified XP approach, it was identified areas of trouble in the entire life cycle, including analysis, design, development, and testing. For each practice which identify, it will discuss the solution implemented to correct it and , more importantly, examine the early symptoms of those poor practices(“bad smells”) that project managers, analysts, and developers need to look out for in order to keep an XP project on its swifter track.

2.1 Smell

Dividing development into “story cards” the way we did last time (with success) is no longer successful.

Solution

With each new development iteration, rethink whether the procedures used in the previous iteration are still appropriate.

2.2 Smell

After product deliveries following early iterations, the customer has no complaint, but during the late iterations, the customer complains about many things from all iterations.

Solution

The customer must be “coached” sufficiently to provide honest and substantial feedback from the very beginning of the development process.

2.3 Smell

When the story cards are written and analyzed, the responsible party for a card feels that he or she cannot be sure all functionality has been accounted for in the functional tests developed for that card.

Password

Confirm Password

I agree to the terms of service and privacy policy

Solution

For complex applications, provide the development team with some synoptic “picture” of the whole that promotes a holistic understanding of the place each story has in the total application.

2.4 Smell

Everyone says that all of the story cards were finished at the proper times, but it still takes 12 more weeks of full-time development effort to deliver a quality application to the customer.

Solution

Create a precise list of tasks that must be completed before the story is considered “finished”, and then police it rigorously and honestly.

2.5 Smell

Incrementally producing several instances of like objects since we ...

This is a preview of the whole essay

Solution

For complex applications, provide the development team with some synoptic “picture” of the whole that promotes a holistic understanding of the place each story has in the total application.

2.4 Smell

Everyone says that all of the story cards were finished at the proper times, but it still takes 12 more weeks of full-time development effort to deliver a quality application to the customer.

Solution

Create a precise list of tasks that must be completed before the story is considered “finished”, and then police it rigorously and honestly.

2.5 Smell

Incrementally producing several instances of like objects since we should do the simplest thing that could possibly work, even though we know that a subsequent iteration will develop a replacement of the several instances with a flexible tool for creating any instance.

Solution

Create a factory instead of building different instances. Look ahead-see where you are going and use common sense. Even if you don’t end up needing the extra flexibility, the cost of design carry is negligible in this case.

2.6 Smell

Basically, the premise here is that if you find yourself having to do a large refactoring then you should have done many smaller refactorings earlier and you got lazy.

Solution

You will still have to do the large refactoring, but you can avoid this in the future by refactoring more often and not putting band-aids on your code.

2.7 Smell

All the unit tests are passing and the system is still broken.

Solution

Automated functional tests. Just like in unit tests, when you fix a bug you need a test to make sure it doesn’t happen again. Many of these tests can be coded and automated right along with the unit tests.

2.8 Smell

Extensive setup and teardown function in unit tests and difficulty setting up complex objects in different parts of their lifetime.

Solution

Create fixtures that will return different objects in different states. It is also a different way to approach the idea of General Fixture presented because this solution becomes more useful in larger and more complex business objects.

Conclusion: Extreme programming is a software development methodology that promotes several desirable practices for a variety of projects. But as much as it may promise greater success in delivering high quality software in a timely fashion, it depends on conscientious people who think carefully about what they do if it is to succeed. In team’s experience, they learned that XP is a valuable and effective approach to software development, so long as one recognizes that 1)it cannot succeed without conscientious participants and 2)it must be adapted as necessary for projects that do not fit the “small team” limits recommended by its founders.

3. Extreme Programming and Database Administration: Problems, Solutions, and Issues

In this part, it will show team’s experience doing just that, how the XP process was or was not used by the DBA team, problems we had on the database side, applied solutions, and solutions that they suggest.

3.1 Problems encountered with an XP development team and a standard DBA team

Lack of unit tests for views/stored procedures: during testing some views/stored procedures did not return the data that they should have. Developers take the database code for granted and did not develop unit test for all the code, at the same time, the database team was fine with writing code, testing it manually, and assuming that it worked correctly. The truth is that both developers and DBAs were at fault.

Code and database are not in synch: One- the actual database constraints were not exactly the same as those of the application. Most of the ‘mismatching’ pieces required code changes that took time and effort to fix during a cleanup stage. Two- it is not possible to ‘forget’ about adding a new column to a table since it will be used by the code. However, it is very easy to ‘forget’ to remove unused tables/columns in such a low-communication environment.

It also some other problems such as Bad database design decisions, Missing database constraint definition, Database changes are a development bottleneck, Database changes have a single owner, Missing communication.

3.2 Solutions to problems encountered

The planning game: the planning game is one of the most important activities that were missed. The team has caught problems/bad designs that could have been avoided if only a member of the DBA team attended the design meetings (can be replaced by a developer with high database design skills). The fact is that not all story cards require DBA input. However, it is very difficult at the planning phase to know exactly how things will touch the database. Therefore they recommend that a member of the DBA team be present in all developer design meetings to be able to point out and discuss changes that need to be made to the database.

Pairing: pairing is applicable only when developing code in the database. The two who should pair are the developer responsible for the card and the DBA responsible for making the needed database changes for that card. This is unconventional pair programming because we have two different skill sets working to solve one problem. Applying a test-first strategy, the DBA should work with the developer when writing the unit tests giving input based on the requirements that were agreed upon in the planning game. Afterwards the developer should pair with the DBA and allow the DBA to drive in writing the stored procedure.

Collective ownership: all code that was developed by pairing a DBA with a developer should be jointly owned. On the other hand, we should distinguish between creating the new database code and applying this new code to all of the environments that need it.

Testing: having unit tests for all DB components will help to discover many issues earlier. To synchronize our database we had manual runs of utilities to find: forgotten-to-remove tables and columns, data type mismatching, and required column constraints.

Conclusion: Database Administration in conjunction with XP is not a frequently addressed subject. The team has presented their experience with a large XP project that included a DBA team and has enumerated their pains. What we do know is that many of the pains that we encountered could have been reduced by the introduction of testing, pairing, DBA involvement in the planning game, and more collective ownership.

4. XP on A Large Project (ATLAS)

In this part, it will introduce this project which is a leasing application which we internally called ATLAS, originally started 3 years ago with the standard analysis and design front-loading of traditional development projects. In this part also conclude the experiences and the techniques that were tried and either became habit because they were useful, or never quite caught on. The team takes the different practices that are encouraged by XP in Kent Beck’s book (Extreme Programming Explained), and give their feedback on each practice.

Element of XP with a large project: the element contain planning, release cycle, metaphor, design simplicity, testing, refactoring, pair programming, collective ownership, continuous integration, 40 hour week, onsite customer, coding standards. Many of these elements we have mentioned on above part. After examining these elements they notice that almost all the practices have evolved over 18 months.

Pair programming: firstly, good communication between developers and iteration planning meetings keep everyone ‘in the big picture’ about who is doing what. Secondly, pair programming is good, but not realistic all the time. The most common reason a developer does not pair program is if he is working on bugs or maintenance. Thirdly, developers have different personalities; it becomes obvious that it becomes a burden for these people.

Unit tests and integrated builds: it is absolutely mandatory- we would stopped in our tracks and not able to deliver one piece of code if we could not rely on tests. The details of every build, tests broken, and people responsible are immediately available on an internal web page that developers can access to see the current state of the build and business analysts and QA can access to retrieve the latest build to test the functionality they are working with.

Group ownership and information sharing: dissemination of information through communication and rotating through different parts of code is important to keep such a large project from fragmenting into several independent pieces that make inaccurate assumptions about the system as a whole.

Short iterations: iterations and deadlines are mandatory- but the length has always been an issue. The team originally had longer iterations- one month iterations- and this caused an end of the month squeeze and bad code being checked in because they had such large cards- and inevitably they had problems estimating. So they are now doing two-week iterations that have made their estimations closer to target and they are also allowing cards to fall through.

Conclusion: So after 18 months on a 50 man team what are their recommendations and lessons learned?

Have an iteration planning meeting at the beginning of each iteration where the customer and developers split up in groups all day to discuss the latest story cards and estimate them.
Keep releases as small as possible.
Write as many unit tests as you can.
Simple designs have helped us release a working product to the customer consistently.
Refactoring is the only way to be able to have simple designs.
Pair programming should be religiously followed when new functionality is added, and should be skipped when fixing bugs or doing repetitive tasks that have already been ‘solved’ before by a pair of developers.
Collective ownership goes hand in hand with communication.
With a large project a group of individuals are need to be the customer- to generate enough work for the large number of developers.
Coding standards have been very informal and this has not been detrimental to our progress.

They also found that the following things didn’t work for them:

Bi-weekly stand-up meetings were not efficient.
Full team meetings during the iteration planning meeting did not work.
1 month long iterations were too long and were detrimental to code quality.
(3) does not work all the time for larger chunks of the code.
Metaphors are unrealistic with large projects.
A 40 hour week has never been an issue for them.

5. ObjectMother Easing Test Object Creation in XP

The ObjectMother pattern’s primary responsibilities are: to create test objects, to tailor those objects at any point during the testing process, and to terminate those objects once testing is complete. When the process of test-writing is made easier and the quality of test data more consistent, developers are likely to write more and better tests.

5.1 ObjectMother and testing

ObjectMother is intended to compliment a xunit testing framework. By relieving test classes of the burden of constructing test objects, ObjectMother allows them to be dedicated to their intended function- testing. The result is better test objects and cleaner test code.

5.2 ObjectMother Basics

ObjectMother manages the lifecycle of test objects, including creation, customization and, when necessary, deletion. The pattern relies largely on two distinct types of methods: creation methods that return all manner of business objects and attachment methods that assist in tailoring the created objects.

5.3 The details

In the sections above, they have referred to ObjectMother as a factory for test objects; this is true, but it is now time to move beyond analogies and get busy with the particulars- specifically, a definition for their pattern, they propose that ObjectMother is an object or set of objects that:

provides a fully-formed business object along with all of its required attribute objects,
return the requested object at any point in its lifecycle,
facilitates customization of the delivered object,
allows for updating of the object during the testing process, and
when required, terminates the object and all its related objects at the end of the test process.

ObjectMother add-ons

The team has yet to address the final point of ObjectMother definition: test object deletion. This is because the best solution here is to avoid having to fiddle with deletion in the first place. If you have architected your application so that it may be run entirely in memory-via the use of POJOs (Plain Ordinary Java Objects) or some other layer of abstraction- then, when desired, the application may be run without ever writing a bit to the database. If nothing is recorded, then nothing needs to be removed, and the final component of their pattern is yours for free. If, however, your application must hit the database while the unit tests are running, you will have to implement a registration method within ObjectMother. Finally, whether or not you need to delete objects at the end of your test run, there are additional services a registration method may provide.

Conclusion: this part has detailed what the team believes are the essential ingredients to be found in any ObjectMother recipe (salt to taste, serve with bread). The three major benefits that the ObjectMother pattern offers are (1) simplified and standardized test object creation, (2) ease of maintenance, because test object creation is entrusted to a specific class or group of classes, and (3) test object clean-up. The one potential downside is the added time spent building the pattern.

6. Conclusion

Ecommerce requires the rapid development of large systems. A lightweight process such as Extreme Programming (XP) is an essential for success. XP is a rigorous approach, mandating frequent regression testing. However, when use of XP with large projects, how does the teams fit in with overall XP team and how or what XP practices are applicable to its responsibilities? I think if we read carefully from examples which we discussed above then we can learn something from it 1) these projects can not succeed without conscientious participants (customers, analysts, developers), 2) we should know the relationship of testing, pairing, collective ownership and refactoring.

References

[1] Amr Elssamadisy, Dr. Gregory Schalliol, Recognizing and Responding to “Bad Smells” in Extreme Programming

International Conference on Software Engineering
Proceedings of the 24th international conference on Software engineering
Orlando, Florida
Pages: 617 - 622
Year of Publication: 2002

[2] Ahmed M. Hassan, Amr Elssamadisy, Extreme Programming and Database Administration: Problems, Solution, and Issues

[3] Elssamadisy, A. XP On A Large project: A Developer’s view

[4] Peter Schuh, Stephanie Punke, ObjectMother Easing Test Object Creation in XP

This paper was originally published by the 2001

XPUniverse Conference.

[5] Cara Taber, Martin Fowler, An Iteration in the Life of an XP Project

Year of Publication: November 2000

[6] Alan Cameron Wills, UML meets XP

Position statement for UML + extremity workshop, UML2001

Trireme International Ltd

[7] Kent Beck, Extreme Programming Explained

Addison Wesley