Java Daemon

Lately I have been writing a Java program that needs to run in the back ground (like a daemon). I found a couple of neat little tricks that can make this easier. These ideas probably only work in a Unix environment but they have been tested on Linux and Solaris.

So you have your program and you want to start it such that it will not be killed when you log out of the shell in which you start it. You could use nohup, but nohup redirects the standard out and error to files, which is annoying because you are writing all output to a log file anyway. You could do java -cp your_class_path com.domain.main_class <&- 1>/dev/null 2>&1 & which runs the program in the background, closes standard in, and redirects standard out and error to the bit bucket. By closing standard in to the process the shell will not kill the program when it exits and it is running in the background so it will not interfere with other actions we might want to perform with this shell.

However, it would be nice if you could print errors that occur during startup to the prompt — for example if the config file were missing. This is nice because it is generally appropriate to sanity check the configuration so that if the program starts there is a significant chance it will actually work correctly. So let’s not redirect standard out and error. That leaves you with
java -cp your_class_path com.domain.main_class <&- &, which works but a shell will not exit while there are still programs attached to its standard out and error — as this one is.

The solution is to have the Java program detach from standard out and error once its startup is complete. So we will create a method called daemonize() that we will call just before entering the infinite loop that is the main program logic.

static public void daemonize()
{
   System.out.close();
   System.err.close();
}

So now we have a main method which looks like

static public void main(String[] args)
{
   try
   {
       // do sanity checks and startup actions
       daemonize();
   }
   catch (Throwable e)
   {
       System.err.println("Startup failed.");
       e.printStackTrace();
   }

   // do infinite loop
}

Now when we start our program we get to see if it started correctly and if it does start it will not be killed when the shell exits nor will it prevent the shell from exiting.

Now that the program is completely detached from the shell the only way to stop it is by killing the process. However, to do that you need to know the pid. Java has no way for a program to figure its pid directly — it is too system dependent. So we will create a shell script to launch our daemon and record its pid for future use.

#!/bin/sh
java -cp your_class_path com.domain.main_class &lt;&amp;- &
pid=$!
echo ${pid} > mydaemon.pid

This script launches the program and then writes the pid to the file ‘mydaemon.pid’. Now when you want to kill the program you can do

kill `cat mydaemon.pid`

There are a couple of problems with this, one is that the pid file gets written even if the daemon failed to start successfully, another is that if the daemon crashes the pid file will still exist — which might lead you to believe it is still working.

We can solve the pid file surviving daemon crash problem by passing it into the Java program like the following

java -Ddaemon.pidfile=mydaemon.pid -cp your_class_path com.domain.main_class &lt;&amp;- &

And the updated the daemonize() method to the following

.

static public void daemonize()
{
   getPidFile().deleteOnExit();
   System.out.close();
   System.err.close();
}

where getPidFile() is a method which returns a File object file specified by the system property “daemon.pidfile”. That way the pid file will be deleted when the VM exits.

For the overly eager creation of the pid file you could add a delay to the shell script and then check to make sure the process is still running before writing the pid file. But how long should the delay be? A better way is to take advantage of the fact the a shell will not exit while a program is attached to its standard out or error — even if the process is running in the background. We know that our program will not detach from those until the startup process is complete. The following shell script achieves this

#!/bin/sh

launch_daemon()
{
  /bin/sh &lt;&lt;EOF
     java -Ddaemon.pidfile=mydaemon.pid -cp your_class_path com.domain.main_class &lt;&amp;- &
     pid=\$!
     echo \${pid}
EOF
}

daemon_pid=`launch_daemon`
if ps -p "${daemon_pid}" &gt;/dev/null 2&gt;&1
then
  # daemon is running.
  echo ${daemon_pid} &gt; mydaemon.pid
else
  echo "Daemon did not start."
fi

This script starts a sub-shell and launches the daemon (in the launch_daemon() function). The sub-shell will only return once the java program has detached from the console — for our program that means it has completed its startup or died. After the launch_daemon() function returns we check to see if the pid it started is still running. If so it means that the daemon started correctly and the we write the daemon’s pid to the pid file. Remember that whenever the daemon’s VM shuts down the pid file will be deleted so you can treat the existence of the pid file as an indication that the process is running.

Now it occurs to you that if a problem occurs during startup you really would like to log it to both the log file and console. Since you are using log4j this is pretty straight forward. Just updated you main method like the following

static public void main(String[] args)
{
   Appender startupAppender = new ConsoleAppender(new SimpleLayout(), "System.err");
   try
   {
       logger.addAppender(startupAppender);
       // do sanity checks and startup actions
       daemonize();
   }
   catch (Throwable e)
   {
       logger.fatal("Startup failed.",e);
   }
   finally
   {
      logger.removeAppender(startupAppender);
   }

   // do infinite loop
}

where “logger” is a static member variable that contains a Logger object. The nice thing about this is you can log message anywhere in you startup code and know that someone will see them, even if it occurs before you have configured the logging based on the application configuration. If the normal application logging is configured, the messages will go both to the console and to the log file for future debugging.

So all that works pretty well. There is another problem though. There is not a clean way to shut this daemon down. We need a graceful way to handle shutdown. So we add the following code to our main class

static protected boolean shutdownRequested = false;

static public void shutdown()
{
   shutdownRequested = true;
}

static public isShutdownRequested()
{
   return shutdownRequested;
}

Then we update our application so that occasionally it checks ‘isShutdownRequested()’ and if it is we leave the main loop. Now our main method looks like

static public void main(String[] args)
{
   Appender startupAppender = new ConsoleAppender(new SimpleLayout());
   try
   {
       logger.addAppender(startupAppender);
       // do sanity checks and startup actions
       daemonize();
   }
   catch (Throwable e)
   {
       logger.fatal("Startup failed.",e);
   }
   finally
   {
      logger.removeAppender(startupAppender);
   }

   while(!isShutdownRequested())
   {
      // wait for stimuli
      // process stimulus
   }
}

This looks pretty good but you still only shutdown from inside the program. The solution is a VM shutdown hook. The is a bit of code that the VM runs when it is shutdown. We create the following method

static protected void addDaemonShutdownHook()
{
   Runtime.getRuntime().addShutdownHook( new Thread() { public void run() { MainClass.shutdown(); }});
}

and update the shutdown method as follows

static public void shutdown()
{
   shutdownRequested = true;

   try
   {
       getMainDaemonThread().join();
   }
   catch(InterruptedException e)
   {
       logger.error("Interrupted which waiting on main daemon thread to complete.");
   }
}

Note that we now wait for the main daemon thread to die. This is because the VM waits for the VM shutdown hooks to complete for exiting but it does not wait for other threads to complete. This join allows the main daemon threads to complete in a controlled way rather than being killed by the VM. Then we update the main method to call this new addDaemonShutdownHook() method

static public void main(String[] args)
{
   Appender startupAppender = new ConsoleAppender(new SimpleLayout());
   try
   {
       logger.addAppender(startupAppender);
       // do sanity checks and startup actions
       daemonize();
       addDaemonShutdownHook();
   }
   catch (Throwable e)
   {
       logger.fatal("Startup failed.",e);
   }
   finally
   {
      logger.removeAppender(startupAppender);
   }

   while(!isShutdownRequested())
   {
      // wait for stimuli
      // process stimulus
   }

   // do shutdown actions
}

Now you can kill the process using kill `cat mydaemon.pid` but shutdown will be orderly and controlled.

So there you have it. A fairly safe and full featured way of create a Unix daemon with Java. Of course, if you do not need the extra control and do not mind have native binaries it might be easier to use Jakarta Daemon.

{Update: corrected a variable name in one of the shell scripts and added a needed closing brace to one of the bits of Java code}

Comments 30

  1. Lalit Kumar wrote:

    Peter

    I just tried your solution and it worked without any problems. I was able to stop the process by killing it as you described. I even closed the command window and it continued to run and that’s what I wanted. Before that I was struggling to find a solution for it on AIX 64bit. I tried a solution from tanukisoftware but it did not run on 64bit version, although it was fine on windows platform. Thanks

    Posted 23 Oct 2008 at 5:40 pm
  2. Chris wrote:

    Thanks for this introduction brought light to my missing knowledge!

    thank dude!

    Cheers Chris

    Posted 24 Oct 2008 at 12:11 am
  3. Albrecht Weiser wrote:

    Peter
    that looks very auspicious for me, so i will try that for a linux-daemon. But there is one thing not clear for me:
    For my understanding of your program i have to implement all my program-logic inside the infinite loop that it works properly to shutdown in a controlled way. If i build a second thread outside the loop, this thread isn’t shutdown save. Is that statement right so far?
    Best regards
    Albrecht

    Posted 30 Dec 2008 at 2:28 am
  4. berendona wrote:

    Thanks a lot. Your post was very helpful. I am using your hint (in a modified way) in my small project.

    Posted 05 Jan 2009 at 7:01 am
  5. Chris Patti wrote:

    I love the post. It really nicely crystalizes the issues around daemons in Java, and made for a nice refresher both for myself and the developer of some otherwise well behaved code that was causing problems in our build process because it was hanging on to stdout and stderr.

    Posted 14 Jan 2009 at 11:22 am
  6. Ozeroff wrote:

    Very useful files search engine. http://myrapida.com is a search engine designed to search files in various file sharing and uploading sites.

    Posted 17 Mar 2009 at 12:19 pm
  7. Albrecht Weiser wrote:

    Hi,
    I implemented everything like described above.
    But when calling my program with ‘java -cp your_class_path com.domain.main_class <&- &’ and close the console it nevertheless quits the launched program. I also tried with ‘-cp your_class_path com.domain.main_class <&- 1>/dev/null 2>&1 &’.
    No way. When i close the terminal-window, the program quits. I debugged my app. It surely steps into the daemonize-method and closes System.err and System.out.
    I’m using a Suse Linux 10.X and connect over SSH (secure shell).
    What’s going wrong there?

    Posted 18 Mar 2009 at 4:52 am
  8. Rick wrote:

    Hi Peter,
    I’m trying to run your program & I’m getting below error:

    getPidFile() & getMainDaemonThread not defined.

    Can you please help me out? I’m new to this daemon program…

    Thanks in Advance!

    Posted 01 Apr 2009 at 12:30 pm
  9. Mar wrote:

    Great post. this definitely fills a gap in the Java curriculum

    Thanks

    Posted 17 Apr 2009 at 12:31 pm
  10. Rick L wrote:

    Hi,
    I want to run the daemon program which will do following:

    1. Get records from postgres database for status=active.
    2. For each record, perform some processing logic & then insert them into another database.

    Now, the question here is if my daemon is done processing all active records, then I really want it in sleep mode rather than using CPU unecessarily.

    My question for Peter is can I make use of your daemon code? If so, how do I take care of not using CPU continously?

    Thanks!

    Posted 05 May 2009 at 3:05 pm
  11. Peter Williams wrote:

    Rick,

    You could definitely do that. Just put a sleep in the infinite loop. The loop would grab all the active records, do what ever it need to do for each of them, then sleep for an appropriate amount of time.

    Posted 05 May 2009 at 9:14 pm
  12. Rick L wrote:

    Hi Peter,
    Thanks for the reply. However, in that case, I’ll have to hard code the time to go in sleep mode, right?
    Also, is it possible that the program will go in infinite sleep & it should wake up when some button from UI is clicked for waking this daemon? If so, how can it be done with your program?

    Thanks!

    Posted 28 May 2009 at 11:54 pm
  13. Raman wrote:

    Hi Peter,
    I’m using ur daemon. But I founf a very big flaw in this code ….Since the loop is forever true, if I restart the database in between, it goes to exception block & then starts logging error to log file. In couple of seconds, the log files goes HUGE & finally 0 disk space!!!

    while(true) is a BAD idea.
    

    Any suggestion for this problem???

    Thanks!

    Posted 25 Jun 2009 at 4:06 pm
  14. Raymond Marfurt wrote:

    Hi Peter,

    I have used your method for years without any problems – thanks so far!!

    Since the latest ubuntu update, the daemon does not start correctly when booting. The daemon starts (I get my log entries, the java app works fine), but then gets terminated after 5 secs.

    I don’t see any information in a log file. Do you have any hint how to analyse this?

    Posted 05 Aug 2009 at 6:33 am
  15. Raymond Marfurt wrote:

    I forgot to mention: while the system is up and running, I can still perfectly start/stop my daemon.

    Posted 05 Aug 2009 at 6:41 am
  16. Arun wrote:

    Peter – Thanks for the tutorial. Very helpful. Here is a missing piece for others.

    static public File getPidFile ()
    {
    return new File(System.getProperty(“daemon.pidfile”));
    }

    Posted 02 Sep 2009 at 12:50 pm
  17. Frank O'Gorman wrote:

    Hi Peter

    I don’t know what shell or terminal you are using, but with bash and a standard xterm closing System.in, System.out and System.err makes no difference to whether a hangup signal is sent to the java process.

    Closing the xterm by clicking the X button causes a hangup signal to be sent. Closing the xterm by typing ‘exit’ or cntrl-D does NOT cause a hangup to be sent. In neither case does closing the input/output make any difference to the behaviour.

    Do you have something other than the hangup signal in mind when you say that the “shell will not kill the program”?

    Posted 09 Sep 2009 at 8:00 am
  18. basu008 wrote:

    Awesome post. Helped me understand all the nuances of daemoning a java class

    thanks

    Posted 06 Dec 2009 at 6:49 am
  19. Sachin Jain wrote:

    Hi

    I am using a client server program for my project. The problem that i am having is that i run the server in the background. It runs perfectly but after some time say 10-12 days even though the program is running which I can see by using the Unix command ps- u project, there is no response from the server. Please can any one tel me what the problem is????. The command that I am using to run the server in the background is :

    nohup command 2>/dev/null 1>/dev/null &

    Thanks

    Posted 11 Jan 2010 at 8:22 am
  20. Jeremy Brooks wrote:

    Good post. The only downside I can see here is that you might miss errors that are dumped to stdout/stderr, for example out of memory errors.

    Posted 19 Jan 2010 at 11:07 am
  21. Jeremy Brooks wrote:

    Sorry, forgot to include my solution for capturing stderr/stdout when needed:

    if (System.getProperty(“detach”).equals(“false”)) {
    // Log message about no detach or something
    } else {
    System.in.close();
    System.out.close();
    System.err.close();
    }

    Now if you are getting weird crashes and expect that you may have an out of memory error or something that is getting printed to stderr, just include the -Ddetach=false system property in your startup command. When you have captured the information you need, remove the system property, or set it to anything other than false, and your program will detach as expected.

    Posted 19 Jan 2010 at 11:27 am
  22. Jeff Shaver wrote:

    What is the solution in the case where the JVM ignores ‘kill ‘ ? TERM, HUP, INT, and QUIT are all ignored. KILL, and USR1 do kill it, but the shutdown() code does not get run.

    Thanks for a very useful article.

    Posted 03 Jun 2010 at 11:51 am
  23. Scott the Werewolf wrote:

    Sounds impressive, but what would you use this java program for? Sorry for asking what may be a silly question…. =P

    Posted 14 Aug 2010 at 11:18 pm
  24. Avi wrote:

    Thanks for this artical. I have learned a lot.

    Posted 27 Jan 2011 at 10:31 am
  25. Ron de Jong wrote:

    Hi Peter,

    Very appreciated all in all, but you’re completely ignoring the main java principle:

    PLATFORM INDEPENDENCY !!!

    Your solution needs to work on:
    Microsoft OS’s to!!! so forget about those shell solutions!!!

    Posted 14 Apr 2011 at 3:46 am
  26. Vineet wrote:

    Good article.

    However, I am seeing the same problem that Jeff Shaver is facing. I am running on Iced Tea JVM on Linux and the shutdown hook does not get executed when I kill the application.

    Anyone else facing this problem?

    Posted 24 Jun 2011 at 1:24 am
  27. Eric wrote:

    I have a working Daemon my problem is when it is run from services (i.e. Server reboots) The Jar is running and responding to status checks but exec() passthrus to Linux do not return or get executed

    Posted 17 Aug 2011 at 2:22 pm
  28. Pablo wrote:

    The DaemonShutdownHook doesn’t hold the app to finalize, you must not finalize the shutdown method till you end the main thread. Why? Because after the shutdown method the Signal continues with System.exit, that means that if you didn’t end your tasks before that, you’re going to leave lines without execution. My way to fix it was:

    static public void shutdown()
    {
    shutdownRequested = true;
    while (!MainClass.endFlag){//put some wait if u want}
    }

    static public void main(String[] args)
    {
    endFlag=false;
    Appender startupAppender = new ConsoleAppender(new SimpleLayout());
    try
    {
    logger.addAppender(startupAppender);
    // do sanity checks and startup actions
    daemonize();
    addDaemonShutdownHook();
    }
    catch (Throwable e)
    {
    logger.fatal(“Startup failed.”,e);
    }
    finally
    {
    logger.removeAppender(startupAppender);
    }

    while(!isShutdownRequested())
    {
    // wait for stimuli
    // process stimulus
    }

    // do shutdown actions
    //end thread shutdown hook
    endFlag=true;
    }

    Posted 26 Aug 2011 at 7:51 am
  29. VaderJM wrote:

    This was a fantastic post. I was able to get a service running in only a couple days (saved me tons of time). Thank you.

    Posted 10 Nov 2011 at 10:48 am
  30. Joao Thomazini Neto wrote:

    Amazing information. Right on target. Many thanks.

    Posted 17 Jan 2012 at 3:46 pm

Trackbacks & Pingbacks 2

  1. From Daemonize a java process » qwerpo on 03 Jun 2009 at 9:30 am

    […] Found this nice article about daemonizing a Java process. […]

  2. From What is Tridium? | Building Automation Monthly on 16 Jul 2012 at 7:33 pm

    […] Daemon- The Daemon processes or as I call them VM background services. Is the functionality to run the background processes that a user does not need to see but are vital for ┬áthe station to function. If you are a geek like me and you want to know more about Java Daemons, you can read about them here and here. […]