"Chester the test-data molester" comes to town

Introduction

"Chester the test-data molester" is a http test server that delivers various error scenarios consistently.

In the p2 project we have the need to test various communication error scenarios, such as when a web server reports illegal last modified dates, reports the wrong file size, endlessly redirects, hits internal server errors, when connection is made via a "hotel style" payment service etc. This was a real pain to set up in an ad-hoc manner, so I decided to create a small equinox based http testserver that is now ready for use.

The testserver has already been invaluable in finding trasport related issues. I thought it was worth writing this short introduction as it may help others test and fix error reporting issues in RCP apps with custom p2 user interface, for those that need to be able to replicate p2 problems where for security/practical reasons it is not possible to access the real repositories, as well as for those that simply need a http server that can create various error scenarios consistently.

How to get it

The testserver resides in the p2 CVS repository - org.eclipse.equinox.p2.testserver. To use it you need to check it out, as well as the org.eclipse.equinox.http bundle. (If you use the p2 team project sets in the p2 releng project you will get everything you need). There is a launch configuration in the testserver project that starts the testserver on "localhost:8080". (You can change the port in the launch configuration if you want).

Basic Services

The testserver starts some basic testing services on the following paths:

• /timeout[/anything] - will wait 10 minutes and then produce no response
• /status/nnn[/anything] - returns html content with the http response status code set to nnn, e.g. /status/500 for an internal server error
• /redirect/nnn[/location] - redirects nnn times and then redirects to location (a path on testserver). If no location given, a html page with a message is generated as the final redirect. Examples:
  • /redirect/3/status/500 - redirects 3 times and then generates a 500 - internal error.
  • /redirect/3 - redirects 3 times and produces a message
  • /redirect/30 - redirects 30 times, and will trigger "too many redirects error" in most configurations
• /never[/anything] - has basic authentication turned on, but will not accept any username/password as valid.

Content Delivery

The testserver also has content available - there is an index.html in the project, as well as some p2 repository test data. The testserver has also "mounted" the eclipse updates 3.4 repository on different paths with different types of wrappers/"molestors" in place.
This set of paths goes to the testserver bundle's web content - i.e. index.html and some p2 test data:

• /public/... - normal access
• /private/... - same as public but requires login with "Aladdin" and password "open sesame"
• /truncated/... - truncates files by delivering less content than stated in length e.g. /truncated/index.html
• /molested/... - returns garbage instead of real content in the later part of the file e.g. /molested/index.html
• /decelerate/... - delivers content chopped up in small packets with delay, e.c. /decelerate/index.html (interesting to watch in firefox which delivers content as it comes in).

This set of paths has mounted http://download.eclipse.org/eclipse/updates/3.4 with various "molestors":

• /proxy/private/... - requires login with "Aladdin" and password "open sesame"
• /proxy/public/... - unmolested access (useful in redirects)
• /proxy/decelerate/... - chops up content and delays delivery
• /proxy/decelerate2/.... - chops up content and delays the last 20% of the delivery
• /proxy/truncated/... - truncates all files
• /proxy/molested/... - generates gibberish for later part of all files
• /proxy/modified/... - delivers various errors in "last modified" (see below)
  
  • .../zero/... - all times are returned as 0
  • .../old/... - all times are very old
  • .../now/... - all times are the same as the request time
  • .../future/... - all times are in the future (which is illegal in HTTP)
  • .../bad/... - the time is not a date at all - the client should throw an error
• /proxy/length/... - delivers various content length errors (see below)
  • .../zero/... - length is reported as 0 (but all content written to stream)
  • .../less/... - less than the correct size is reported (all content written)
  • .../more/... - double the correct size is reported (but only available content is written)

Get in touch

I had fun writing this - I mean, how often do you get to write classes called "Molestor" :). I hope you find the testserver useful, and that if you would like it to perform other forms of content maiming and mutilation that you contribute by submitting patches to the "p2" project marking issues with the text [testserver].

Progress Monitor Patterns

Are you working with Progress Monitors, and wish you did not have too?

I recently got to investigate issues in p2 why the progress bar did not move while it was very clear that things where happening as subtasks changed the text. Looking into what was going on made me find a new Progress Monitor anti pattern.

The problem is how to handle a case where you need to do something like this:

for (int i = 0; i < candidates.length; i++)       if (loadCandidate1(i))         break;

Where a call to loadCandidate is potentially a long running task. The first loaded candidate means we are done.

Antipattern

Here is an implementation of the above example using the antipattern - i.e. "don't do this:

public void antiPattern(IProgressMonitor monitor) {     SubMonitor sub = SubMonitor.convert(monitor, candidates.length);     for (int i = 0; i < candidates.length; i++) {       if (loadCandidate(i, sub.newChild(1)))         break;     }   }   public boolean loadCandidate(int index, IProgressMonitor monitor) {     SubMonitor sub = SubMonitor.convert(monitor, 1000);     try {       for (int i = 0; i < 1000; i++)         sub.worked(1);       // ...       return true;     } finally {       monitor.done();     }     return false;   }

Well, what is wrong with this, you may ask… Well, the length of the progress bar will be divided into as many slots as there are candidates, and if the first candidate succeeds and uses its 1000 ticks, and the remaining candidates are never considered, we will end up reporting the 1000 ticks on a fraction of the overall progress bar. This means that for 10 candidates, you will see the progress-bar slowly go to about 10% of the overall length, to suddenly jump to 100%.

Good pattern

Here is the good pattern that makes use of the full progress bar:

public void goodPattern(IProgressMonitor monitor) {     SubMonitor sub = SubMonitor.convert(monitor, 1000);     for (int i = 0; i < candidates.length; i++) {       sub.setWorkRemaining(1000);       if (goodLoadCandidate(i, sub.newChild(1000)))         break;     }   }   public boolean goodLoadCandidate(int index, IProgressMonitor monitor) {     SubMonitor sub = SubMonitor.convert(monitor, 1000);     sub.beginTask(null, 1000);     try {       // … code that loads       for (int i = 0; i < 1000; i++)         sub.worked(1);       return true;     } finally {       // ignore errors     }     return false;   }

Notice how “setWorkRemaining” is called each time in the loop. This reallocates the remaining ticks, but there is no need to compute how many that actually remains, the child allocation of 1000 ticks will always give the child 1000 ticks to report, even if there were not enough ticks left in the parent. All the scaling is performed by the SubMonitor, so you don’t have to worry about it.

Now, let’s say that the routine you are calling do need to perform a bit of work even if it does not need all of its ticks. Don’t worry, that will work too as long as it is a small portion of the allocation. If you risk consuming a larger part, you may be better off doing a true partitioning of the progress-bar as shown in the next section.

Good pattern - regular loop

Here is the good pattern for a regular loop where each iteration does consume ticks

public void goodLoopPattern(IProgressMonitor monitor) {     SubMonitor sub = SubMonitor.convert(monitor, candidates.length*100);     for (int i = 0; i < candidates.length; i++) {       if (goodLoopCandidate(i, sub.newChild(100)))         break;     }   }   public boolean goodLoopCandidate(int index, IProgressMonitor monitor) {     SubMonitor sub = SubMonitor.convert(monitor, 1000);     sub.beginTask(null, 1000);     try {       // … code that loads       for (int i = 0; i < 1000; i++)         sub.worked(1);       return true;     } finally {       // ignore errors     }     return false;   }

Here I simply use SubMonitor’s newChild, this works without calling “done” because the next call to newChild (or “done” for that matter) will consume the ticks allocated for the child.

Rules of Thumb

1. Use SubMonitor
2. Always begin by converting the monitor you get to a SubMonitor
3. Use newChild(n) to create a sub monitor to pass to methods that take an IProgressMonitor as argument.
4. Never call “done” - but document that the caller must do so unless they used a SubMonitor