I have a relativly simple select query which contains a distinct. Something like “select distinct id, content from table”. Nothing fancy so you’d expect the content of the table excluding all duplicates.

But this is also where the weirdness begins with MS Access. I was using this type of query to select data for my search engine. But some data wasn’t included for some reason.

After a lot of debugging my queries and code it turned out that MS Access ODBC was only returning the first 255 characters of a MEMO field. So I thought the driver was to blame, until I ran the same query in MS Access itself. As it turns out Access was the one truncating MEMO fields to VARCHAR(255) fields when using a DISTINCT in a query.

Note: some parts of this script rely on the ant-contrib library. Download this library and include it in the tomcat.xml with the following code:

<taskdef resource="net/sf/antcontrib/antcontrib.properties">
 <classpath>
  <pathelement location="./ant-contrib-1.0b3.jar"/>
 </classpath>
</taskdef>

Starting tomcat

Though starting tomcat may seem easy. You could just use the development IDE (like Netbeans) to start and stop Tomcat, but what this lacks is the ability to control several instances with easy shortcuts. So I defined the following Ant macrodef in a file called tomcat.xml:

<macrodef name="tomcat-start">
 <sequential>
   <trycatch>
    <try>
      <if>
       <not><http url="http://localhost"/></not>
       <then>
         <java classname="org.apache.catalina.startup.Bootstrap"
             fork="yes"
             dir="${tomcat.dir}"
             spawn="true"
             jvm="${tomcat.java.home}/bin/java">
          <jvmarg value="-Dcatalina.home=${tomcat.dir}"/>
          <jvmarg value="-Dcatalina.base=${tomcat.dir}"/>
          <jvmarg value="-Djava.io.tmpdir=${tomcat.dir}/temp"/>
          <jvmarg value="-Djava.endorsed.dirs=${tomcat.dir}/common/endorsed"/>
          <jvmarg value="-Xdebug"/>
          <jvmarg value="-Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n"/>
          <classpath>
           <pathelement location="${tomcat.java.home}/lib/tools.jar"/>
           <pathelement location="${tomcat.dir}/bin/bootstrap.jar"/>
          </classpath>

          <arg line="start" />
         </java>

         <waitfor maxwait="10" maxwaitunit="second" checkevery="5000">
          <http url="http://localhost"/>
         </waitfor>
         <echo message="Tomcat started"/>
       </then>
       <else>
        <echo message="Tomcat already started..." />
       </else>
     </if>
    </try>
    <catch>
     <echo message="Unable to start tomcat"/>
    </catch>
  </trycatch>
 </sequential>
</macrodef>

This macro is really easy. First it starts of by checking if tomcat is not already running (well actually it just checks to see if there is something running on port 80). If nothing is running then tomcat is started using the java defined by the ant property tomcat.java.home. This macro also depends on some other properties that are set by a different macro, which will be a bit further on in the post. Tomcat is started by using the Bootloader class provided by the server, to make sure it loads correctly the classpath is set.

The last step in the macro is to wait for tomcat to start, this has to be done since we start tomcat in a seperate java instance (the spawn option).

Stopping Tomat

Off course you also want to be able to stop tomcat to load some new data, or an entire web-app. I use the following macro to stop Tomcat:

<macrodef name="tomcat-stop">
 <sequential>
  <trycatch>
   <try>
    <java classname="org.apache.catalina.startup.Bootstrap"
         fork="yes"
         dir="${tomcat.dir}"
         spawn="true"
         jvm="${java.home}/bin/java">
     <jvmarg value="-Dcatalina.home=${tomcat.dir}"/>
     <jvmarg value="-Dcatalina.base=${tomcat.dir}"/>
     <jvmarg value="-Djava.io.tmpdir=${tomcat.dir}/temp"/>
     <classpath>
      <pathelement location="${tomcat.java.home}/lib/tools.jar"/>
      <pathelement location="${tomcat.dir}/bin/bootstrap.jar"/>
     </classpath>

     <arg line="stop" />
    </java>

    <kill-java name="Bootstrap" />
    <echo message="Tomcat stopped" />
   </try>
   <catch>
    <echo message="Unable to stop tomcat forcing shutdown....."/>
    <kill-java name="Bootstrap" />
   </catch>
  </trycatch>
 </sequential>
</macrodef>

Just like with the starting of Tomcat I use the provided Bootloader class to instruct tomcat to stop. Again I use the java that is set in tomcat.java.home as well as some other properties loaded by another macro. Since Tomcat sometimes fails to stop gracefully (due to poorly designed webapps) you also have to kill the Java process of Tomcat. This is done with the call to kill-java.

Killing any java process

As you saw in the tomcat-stop macro I used a macro called kill-java to make sure that Tomcat is really killed and no longer running in the background. The macro is as follows:

<macrodef name="kill-java"
          description="Forcefully stop tomcat....">
 <attribute name="name"/>
 <sequential>
  <!-- Execute the jps and check for any Java process with the provided @{name} attribute -->
  <exec executable="${tomcat.java.home}/bin/jps" output="pid.out.file" />
  <!-- Load in the name / pid file and strip all information except the PID -->
  <loadfile srcfile="pid.out.file" property="pid.out">
   <filterchain>
     <linecontains>
      <contains value="@{name}"/>
     </linecontains>
    <tokenfilter>
    <deletecharacters chars="@{name}"/>
    <trim/>
    <ignoreblank/>
    </tokenfilter>
    <striplinebreaks/>
   </filterchain>
  </loadfile>
  <echo message="Killing java process with pid ${pid.out}"/>
  <!-- Kill the process, warning this only Works on Windows -->
  <exec spawn="true" executable="taskkill">
   <arg line="/PID ${pid.out}" />
   <arg line="/F" />
  </exec>
  <delete file="pid.out.file" />
 </sequential>
</macrodef>

This macro is really simple and relies on the jps application provided by Java. This application returns a list of all running processes with name and process id (PID). All we need to do is get the line containing the process name provided and strip everything except the PID.

Please note: the task killing the proces is designed for Windows, you could change this with kill in Linux.

Rounding it up

Though you now have all you need to manipulate tomcat there is one last macro that we are tegenkant upon. That is the initializer of the various Tomcat properties. I use the following macro:

<macrodef name="tomcat-init">
 <attribute name="from"/>
 <sequential>
  <property name="tomcat.dir" value="${tomcat.@{from}.dir}"/>
  <property name="tomcat.server" value="${tomcat.@{from}.server}"/>
  <property name="tomcat.port" value="${tomcat.@{from}.port}" />
  <property name="j2ee.server.type" value="${tomcat.server}" />
  <taskdef name="webapp-stop"
          classname="org.apache.catalina.ant.StopTask"
          classpath="${tomcat.dir}/server/lib/catalina-ant.jar"/>
  <taskdef name="webapp-start"
           classname="org.apache.catalina.ant.StartTask"
           classpath="${tomcat.dir}/server/lib/catalina-ant.jar"/>
 </sequential>
</macrodef>

This little macro will initialize tomcat for a specific environment (for exemple DEV). Of course none of this works without the loading of some preset properties in a property file tomcat.properties. Which contains the following data:

tomcat.java.home=C:/jdk1.5.0_22_32b

tomcat.DEV.dir=d:/jakarta/tomcat5.0
tomcat.DEV.server=tomcat55-DEV
tomcat.DEV.port=8000

tomcat.MAIN.dir=d:/jakarta/tomcat5.0-MAIN
tomcat.MAIN.server=tomcat55-MAIN
tomcat.MAIN.port=8000

To load the property file just include the following instruction in the build script:

<property file="./tomcat.properties" />

The reason for me not to write one straight away was simple, I first wanna play around a little to find all the pleasant features and those less pleasant ones.

Bigger, better, best

The first thing you expect is this new version to be bigger, better and the best one yet. Well it is bigger in size, with 190MiB for just Java development. So this means a lot more time downloading, but hey that’s a one time thing.

So lets move onto something more relevant. Is it better then Netbeans 6.1. Lets summorize some of the issues I had with the previous version and see if the problem is solved:

1. It crashed when working on larger projects. This issue has been fixed, netbeans won’t crash or hang. Though on occasion the parser may crash. Disabling autocomplete and syntax checking.
2. For some reason previous versions had difficulty displaying the names of parameters when using Java libraries. For as far as I’ve been able to determine this has been solved.
3. The autocomplete completely fails in some more complicated web projects. Well I’m sorry to say this is still true. For some projects not only does autocomplete fail, but so does the syntax checker, both for the JSP files as well as the Java Source files.

So are there any other changes that would make you consider moving from Netbeans 5.5 to 6.5?

New stuff you will like in Netbeans 6.5

Well have there been any changes that would make you wanna switch to the new version.

• The new version actually has some basic debugging integrated of JavaScript. Though I haven’t got it working yet.
• PHP development is now support. Which is a big plus, but it still kinda feels like a cheap addon that doesn’t work perfectly yet. (Maybe in the next release)
• Thread debugging has been improved and made easier

Unfortunattely not all is good in the world of Netbeans. There are also some new bugs, some of which are really annoying.

• For some unexplainable reason adding of libraries by right clicking the ‘Libraries’ in the project view doesn’t work all the time.
• Like I stated earlier the code completion fails in some of the older Java project I’m working on. Not only does code completion fail, but the syntax checker tells me every single line is wrong. Even though the project compiles beautifully

As a final note: is it worth upgrading to this version. Then I’d have to say hell yeah, it’s a lot better then previous versions. If you are a PHP developer however I suggest not using it, for the moment. Especially when you are already using Zend-Studio, which is still a lot better.

First things I was seriously hoping was that they fixed the bug with Netbeans not recognizing the names of function parameters of libraries. This is a problem which got me so pissed that I switched back from Netbeans 6 to 5.5, which did not have the problem. Both Netbeans 6.0 and Netbeans 6.1 had the same problem with function parameters.

Let’s just say that on this they did not disapoint. The new Beta version has got no issues recognizing the names of function parameters. So that’s a serious improvement of the previous releases. I also like the new style for debugging and code editing.

The only real issues so far are the fact that with large web applications Netbeans 6.5 constantly hangs and stops responding. Another issue is that the debugger does not highlight the line that it’s on. The latter is probably a simple fix, but it does make debugging slightly difficult.

So lets wait and see if some of these issues get resolved in the final release.

Some time back I already wrote about the setup we use in counting sessions. Read more on that in the ‘Cleaning Up Java Sessions’ post. We put live this change some time back. But I’m still having some issues with the sessions.

What we do is something like this. When a user comes in through a servlet we look if a specific object is set in the session. If not we invalidate the session and add the object. In a seperate class we catch all session starts and destructions.

Upon the session start function in HttpSessionListener we increase the counter. Upon the destruction of the session and if the our session object is set we decrease the counter. But for some reason we have some sessions that do not get cleaned up by Tomcat. Or there are some users that simply stay online for over 2 months.

To get a semi accurate count of the active session I looked into getting the amount of active sessions from the Tomcat StandardManager class. In theory you would be able to obtain this through the tomcat implementation of the HttpSession object. But for some reason these objects are package protected.

There has to be an easier way of getting an accurate count of the amount of active sessions through the tomcat API.

Once a session became invalidated the HttpSessionBindingListener valueUnbound method was called. During this call we used the getLastAccessedTime() on the session to figure out how long the session was idle before the session was cleaned up.

During the investigation as to why the idle time was no longer logged we found out that for some reason the valueUnbound was no longer working the same in Tomcat as it used to in Resin. After some debuging it appeared as if the HttpSession.getLastAccessedTime() could no longer be used at this stage of the clean up. Every time you tried to access it I got a IllegalStateException.

Some searching on the web later toled me that this was because the session is already invalidated at this point. And an invalidated sessions information cannot be trusted. So what I had to do is build another listener. This time one that is more generic and catches all session deletions. The class to implement in order to do this is called the HttpSessionListener. The methods on this class are called when a session is made or destroyed.

By storing the last access time during the destruction the idle time is available.

But when you want to look at files in the file explorer you might have some trouble finding the right file, and you may find expanding all the directories annoying (I know I did ). Especially when you are browsing the SVN or CVS when each folder takes forever to load .

As I noted in the first paragraph were over a year further and I must say that I like structuring the folder structure the same as the namespaces in the code. Let me explain why.

Easier to find a class
Though you may not notice it at first, but it is easier to find a class when the directory structure is similar to that in the source code. After all you already know the namespace it’s in. So locating the files becomes a lot easier.

You instantly know what classes belong to which namespace
I don’t know about all off you but I don’t always have my projects open when I’m documenting on trac. So to figure out which classes are part of a namespace I simply open the directory and find out. That way I don’t have to load the entire project every single time.

Moving of classes to other namespaces becomes easier
When you come to the point that you need to refactoring having all classes from a namespace in one directory makes it a lot easier to refactor. No longer do you have to open each file to find out in which namespace it’s located, just look at the directory structure.

This also introduces the downside that you need to move the classes in the SVN or CVS since the namespace has changed, but hey everything has it’s downsides .

Let me know if you now any good reasons, or have some strong reasons against.