Skip to content

Monitoring mounts

August 6, 2009

Sometimes in the course of being a system administrator it is useful to know what file systems are being mounted and when and what mounts fail and why. While you can turn on automounter verbose mode that only answers the question for the automounter.

Dtrace makes answering the general question a snip:

: exdev.eu FSS 24 $; cat mount_monitor.d                          
#!/usr/sbin/dtrace -qs  
fbt::domount:entry / args[1]->dir / 
{         
     self->dir = args[1]->flags & 0x8 ? args[1]->dir :                copyinstr((intptr_t)args[1]->dir); 
} fbt::domount:return 
/ self->dir != 0 / 
{                  
         printf("%Y domount ppid %d, %s %s pid %d -> %s", walltimestamp,
            ppid, execname, self->dir, pid, arg1 == 0 ? "OK" : "failed"); 
} fbt::domount:return 
/ self->dir != 0 && arg1 == 0/ 
{         
         printf("\n");
         self->dir = 0; 
} 
fbt::domount:return 
/ self->dir != 0 && arg1 != 0/ 
{         
         printf("errno %d\n", arg1);
         self->dir = 0; 
} : exdev.eu FSS 25 $; pfexec /usr/sbin/dtrace -qs  mount_monitor.d 
2009 Aug  6 12:57:57 domount ppid 0, sched /share/consoles pid 0 -> OK 
2009 Aug  6 12:57:59 domount ppid 0, sched /share/chroot pid 0 -> OK 
2009 Aug  6 12:58:00 domount ppid 0, sched /share/newsrc pid 0 -> OK 
2009 Aug  6 12:58:00 domount ppid 0, sched /share/build2 pid 0 -> OK 
2009 Aug  6 12:58:00 domount ppid 0, sched /share/chris_at_play pid 0 -> OK 
2009 Aug  6 12:58:00 domount ppid 0, sched /share/ws_eng pid 0 -> OK 
2009 Aug  6 12:58:00 domount ppid 0, sched /share/ws pid 0 -> OK 
2009 Aug  6 12:58:03 domount ppid 0, sched /home/tx pid 0 -> OK 
2009 Aug  6 12:58:04 domount ppid 0, sched /home/fl pid 0 -> OK 
2009 Aug  6 12:58:05 domount ppid 0, sched /home/socal pid 0 -> OK 
2009 Aug  6 12:58:07 domount ppid 0, sched /home/bur pid 0 -> OK 
2009 Aug  6 12:58:23 domount ppid 0, sched /net/e2big.uk/export/install/docs pid 0 -> OK 
2009 Aug  6 12:58:23 domount ppid 0, sched /net/e2big.uk/export/install/browser pid 0 -> OK 
2009 Aug  6 12:58:23 domount ppid 0, sched /net/e2big.uk/export/install/cdroms pid 0 -> OK 
2009 Aug  6 12:59:45 domount ppid 8929, Xnewt /tmp/.X11-pipe/X6 pid 8935 -> OK

In particular that last line if repeated often can give you a clue to things not being right.

Advertisements

From → Solaris

5 Comments
  1. Karl permalink

    Gave the script a try on our sunray. Noticed Xnewt restarting

    2010 May 11 13:29:09 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8218 -> OK
    2010 May 11 13:29:35 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8323 -> OK
    2010 May 11 13:30:00 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8347 -> OK
    2010 May 11 13:30:25 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8372 -> OK
    2010 May 11 13:30:50 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8390 -> OK
    2010 May 11 13:31:15 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8420 -> OK
    2010 May 11 13:31:40 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8442 -> OK
    2010 May 11 13:32:05 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8466 -> OK
    2010 May 11 13:32:31 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8483 -> OK
    2010 May 11 13:32:56 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8505 -> OK
    2010 May 11 13:33:21 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8523 -> OK
    2010 May 11 13:33:46 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8541 -> OK
    2010 May 11 13:34:11 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8563 -> OK
    2010 May 11 13:34:36 domount ppid 956, Xnewt /tmp/.X11-pipe/X5 pid 8579 -> OK

    I assume that’s a DTU with a 26d. I’m not sure what else to check now.

    • chrisgerhard permalink

      The trick here is to modify the script to stop the process when you see these loops. That at least stops the process from eating your system.

      Then call sun/oracle to get them/us to investigate further. I have seen this and there is a bug already filed but I can’t recall the number now. I’ll dig it out tomorrow.

  2. Karl permalink

    Can’t place a service call since the server is running OpenSolaris B134. I have 3 (FOG) srs 4.2 servers running on 3 xen vm’s.

    I’m not sure how much further away from the support matrix i can get 😉

  3. chrisgerhard permalink

    Are you using a UNIX desktop or displaying windows?

  4. Karl permalink

    110 dtu’s connect to the 3 xen images each running srs 4.2 in a fog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: