Building gcc4 with gcc3
Compiling gcc4 on an system with only old gcc versions available, can be tricky. Here is an HowTo for compiling gcc4.4 on Solaris 10, which only ships 3.4.3. It will be similar for other systems with an older gcc versions.Monitoring Postgres autovacuum
Postgres does all updates on a database Copy-on-Write. That means, on every update on a row, additional space is needed on the filesystem. If noone is using an old entry anymore, the space allocated by these data is not freed automatically, nor will it be available for postgres to store new data. To get the space back, you have to vacuum, luckily there is an autovacuum in postgres. Autovacuum is an extra deamon that will run vacuum from time to time. It will not run a vacuum full, thus it will never free disk space, but postgres is able to reuse the already allocated space and thus the datafiles will not grow infinitely.
The cumbersome part is to check if autovacuum is working, since there is no way to ensure that it will do. First you can check, if autovacuum is configured to run. You might think setting "autovacuum = on" would be enough, it's not, since it will only work with some version depending statistics enabled.
Potstgres >=8.2
SHOW autovacuum; SHOW track_counts;
Prior versions:
SHOW autovacuum; SHOW stats_start_collector; SHOW stats_row_level;
Next is to check, if the autovacuum daemon is running. There is one postgres process that will be named "postgres: autovacuum launcher process". If you are running solaris, ps will not show you that name, instead it will show the original process cmdline, since it is ignoring updates on argv[0] made by processes at runtime. You can help your self with a little loop.
for PGPR in `/usr/bin/pgrep postgres`; do pargs -a $PGPR; done | grep 'argv\[0' | sed 's/ *$//'
An more easy and generic way is to look into the logfile, if you've not already deleted the log containing the startup messages, you'll find the following line there:
LOG: autovacuum launcher started
You can also monitor the activity of autovacuum on the database. However, it will only show you activities if autovacuum had done something, thus on an fresh installed database it is very unlikely to see any activity.
postgres=# SELECT relname,last_autovacuum from pg_stat_user_tables;
relname | last_autovacuum
-------------------------+-----------------
my_table | 2011-05-02 05:05:11.78106+02
my_other_table | 2011-05-03 09:21:18.850887+02
unlikely_for_update |
If you select pg_stat_all_tables instead of pg_stat_user_tables you will also see vacuums for internal tables.
drace syscall times
Sometimes processes are slow without consuming much ressources. In such cases you can try to use truss (or strace on some systems) to find out what the process is waiting for, however, in multithreaded processes this might not help you that much. Also tracing the process from userland will slow down the process further what might not be an option in production einvironments.
Using dtrace will not only be less intrusive to your application flow, it can also summarize the times for you. The following script will not only summarize the times spending in system calls (/opt/DTT/procsystime would do), it will also print you a line for each system call when it retruns
#!/usr/sbin/dtrace -s
#pragma D option quiet
syscall:::entry
/ zonename == "myzone" && execname == "tntnet" /
{
self->t=timestamp;
}
syscall:::return
/ zonename == "myzone" && self->t!=0 && execname == "tntnet" /
{
printf("%d/%d spent %d nsecs in %s\n",
pid, tid, timestamp - self->t, probefunc);
@function[probefunc] = sum (timestamp - self->t);
}
For these who are familiar with awk, the syntax will not surprise you, in / / there are filters on which the code blocks apply, self is a special variable you can use to ensure coherency between threads.
... 14611/52 spent 3415416747 nsecs in pollsys 14611/52 spent 8583 nsecs in gtime 14611/52 spent 5333 nsecs in gtime 14611/52 spent 21666 nsecs in lwp_park 14611/52 spent 2798833 nsecs in nanosleep 14611/52 spent 20010167 nsecs in pollsys 14611/52 spent 18000 nsecs in read 14611/52 spent 9864917 nsecs in nanosleep 14611/52 spent 5417 nsecs in gtime 14611/53 spent 10008364573 nsecs in lwp_park 14611/53 spent 11500 nsecs in gtime 14611/53 spent 5500 nsecs in gtime ^C lseek 29583 fcntl 34666 close 45083 accept 57417 getpid 133751 open 134834 stat 248915 gtime 282585 read 1725499 write 2817832 nanosleep 80435583 pollsys 5699637580 lwp_park 35622920131
You see, that the process is spending the most time on beeing idle and waiting for requests, but tntnet is actually not a really good example since it doesn't appreas to be slow.
Solaris ptools
Solaris has some features, many other systems are missing. One of my favorites is the large number of process tools. While i could write the whole day about ptools, i like to concentrate on my favorites here. Some of them may also be are available on other plattforms.
pldd
pldd acts like ldd but on processes, thus you see which librarys are really loaded by the program. Yes there might are differences to the binaries on the filesystems.
$ pldd 1259 1259: /home/jw/bin/ncmpc /usr/lib/libglib-2.0.so.0.2501.0 /lib/libsocket.so.1 /lib/libnsl.so.1 /usr/gnu/lib/libncurses.so.5.7 /usr/lib/libc/libc_hwcap1.so.1
pargs
pargs shows arguments -a and environment variables -e of an running process.
$ pargs -ea 284 284: /usr/lib/inet/ntpd -p /var/run/ntp.pid -g argv[0]: /usr/lib/inet/ntpd argv[1]: -p argv[2]: /var/run/ntp.pid argv[3]: -g envp[0]: _=*213*/usr/lib/inet/ntpd envp[1]: LANG=en_US.UTF-8 envp[2]: PATH=/usr/sbin:/usr/bin envp[3]: PWD=/root envp[4]: SHLVL=1 envp[5]: SMF_FMRI=svc:/network/ntp:default envp[6]: SMF_METHOD=start envp[7]: SMF_RESTARTER=svc:/system/svc/restarter:default envp[8]: SMF_ZONENAME=global envp[9]: TZ=Europe/Berlin envp[10]: A__z="*SHLVL
pstack
pstack prints a strack trace from an running process.
$ pstack 11717 11717: ksh ceb04875 waitid (7, 0, 8046f70, f) ceaa00a5 waitpid (ffffffff, 804703c, c, ce99076f) + 65 ce9907c8 job_reap (0, ce9f7000, 80470a8, ce99287c) + c0 ce992af7 job_wait (3c82, 806a8a8, 0, 8047130) + 4ab ce9c8ae3 sh_exec (806755c, 4, 0, ce9a0f3d) + 43af ce9a10e6 exfile (ce9f7bb0, ce8d0fa8) + 732 ce9a0920 sh_main (1, 8047964, 0, cea5ea42, 4, 0) + 830 08050bd6 main (1, 8047964, 804796c, 8050a9f) + 4a 08050afd _start (1, 8047a7c, 0, 8061210, 8061230, 8061248) + 7d
plimit
plimit works like ulimit on processes. Remarkable is, that it doesn't only print the limits of an running process, you can also modify the limits without restarting the process.
$ /usr/bin/plimit 11716 11716: xterm resource current maximum time(seconds) unlimited unlimited file(blocks) unlimited unlimited data(kbytes) unlimited unlimited stack(kbytes) 8480 130336 coredump(blocks) unlimited unlimited nofiles(descriptors) 256 65536 vmemory(kbytes) unlimited unlimited $ /usr/bin/plimit -n 1024 11716 $ /usr/bin/plimit 11716 11716: xterm resource current maximum time(seconds) unlimited unlimited file(blocks) unlimited unlimited data(kbytes) unlimited unlimited stack(kbytes) 8480 130336 coredump(blocks) unlimited unlimited nofiles(descriptors) 1024 65536 vmemory(kbytes) unlimited unlimited
cacheControl for tntnet
tntnet contains an static component, that allows you to serve static content directly from your filesystem using send-file. But the static component doesn't add Expire or cacheControl headers, thus clients or proxys will ask every time if the file as changed. For files like your CSS what will likely be loaded for every site but changes only seldom, this will significantly increase the number of unnecessary requests to tntnet.
Luckily tntnet allows you to pass a requests to more than one component, thus you can benefit from the easiness of the static componend and add additional haeders.
<%pre> #include <tnt/httpheader.h> </%pre> <%cpp> reply.setHeader(tnt::httpheader::cacheControl, "public, max-age=86400"); return DECLINED; </%cpp>
The return DECLINED will cause tntnet to keep whatever content the component added to reply and then pass the request to the next componend on the UrlMap.
MapUrl ^/static/(.*) expire@vtoc MapUrl ^/static/(.*) static@tntnet static/$1
cxxtools regex
cxxtools contains a nice class that allows you to use regular expressions in C++. Since it is so handy and not very documented, i like to show an short example.
#include <string>
#include <cxxtools/regex.h>
bool TwoFour (const std::string& word)
{
cxxtools::Regex checkTwoFour("^[aA-zZ]{2,4}$");
return checkTwoFour.match(word);
}
The function will return true only if word is between 2 and 4 characters long and consist only of alphabetic characters. Of course you can use more compley regular expression.
non-root SMF service
Many services doesn't require to run as root. Traditionally most services do start as root and just drop their privileges. With Solaris (>=10) you don't need that anymore, you can start services directly as user from the SMF.
Some Services like Webserver need "root privileges" at start to bind port 80 for example. However, in Solaris there are no "root privileges". There are many privileges now, one privilege called "net_privaddr" is excactly the privilege a Webserver will need to bind port 80, so there is no need to give all privileges a root user normaly have to the webserver.
<exec_method
type='method'
name='start'
exec='/usr/local/bin/tntnet -c /var/tntnet/tntnet.conf'
timeout_seconds='60'>
<method_context>
<method_credential user='tntnet' group='tntnet' privileges='basic,net_privaddr'/>
</method_context>
</exec_method>
This is an example from my tntnet SMF manifest, you can download the full manifest here.
vtoc.de
Back in 2005 just everyone had a webblog, at least most of my friends. Today the people using Twitter or Facebook to let everyone know whatever they want them to know, and noone cares about the old weblogs. So it's the right time to start one.
Actually this is how the story could be told but it is not why i'm starting this blog. This blog is much more a software experiment rather than a blogging experiment. I wanted to start experimenting with C++ again since i stopped it arround 8 years ago. You might argue that starting a blog doesn't have mucht to do with C++ experiments, but this blog actually has, since the blog software is entirely written in C++ using tntnet.
So, it is a programming experiment, and i like to try the code in practise. I'll add some content about programming, tntnet, Unix and may be some other technical stuff here, since i need content and readers do get any practical resuls from this experiment.