The Shœstring Foundation Weblog

The Shœstring Foundation Weblog, Miscellaneous Byproducts

Matthias Bauer
bauerm (at) shoestringfoundation · org
reop pubkey
Vignettes by George Herriman and a small program

Subscribe to a syndicated feed of my weblog, brought to you by the wonders of RSS.

Blosxom Logo

Fri, 05 Apr 2013

Programming Pearls: malloc, the Zabbix way

The Zabbix sources define zbx_malloc in include/common.h: line 700 as:

#define zbx_malloc(old, size)   zbx_malloc2(__FILE__, __LINE__, old, size)

zbx_malloc2 is defined in src/libs/zbxcommon/misc.c: line 255 as:

void    *zbx_malloc2(const char *filename, int line, void *old, size_t size)
    int max_attempts;
    void    *ptr = NULL;

    /* old pointer must be NULL */
    if (NULL != old)
        zabbix_log(LOG_LEVEL_CRIT, "[file:%s,line:%d] zbx_malloc: " 	
				"allocating already allocated memory. "
                "Please report this to Zabbix developers.",
                filename, line);
        /* exit if defined DEBUG, ignore otherwise */

    for (
        max_attempts = 10, size = MAX(size, 1);
        0 < max_attempts && NULL == ptr;
        ptr = malloc(size), max_attempts--

    if (NULL != ptr)
        return ptr;

    zabbix_log(LOG_LEVEL_CRIT, "[file:%s,line:%d] zbx_malloc: out of memory."
		"Requested " ZBX_FS_SIZE_T " bytes.",
            filename, line, (zbx_fs_size_t)size);


So where're the pearls in this?

  1. the old parameter is forced to be NULL, so why pass it at all???
  2. noticed the for loop? It tries to malloc ten times before giving up. This seems to assume that some concurrently running part of zabbix frees memory, or that the system suddenly (while this loop is running, that is) assigns a higher memory bound for zabbix.
  3. noticed the MAX in the for loop? It tries to catch and disguise the error of requesting zero bytes by always returning at least one byte

[/osfail] permanent link

Sat, 04 Feb 2012

OpenStreetMap's pseudo XML format

OpenStreetMap's pseudo XML .osm format is wrong on so many levels that you'd have to see it for yourself (excerpt from mittelfranken.osm)
 <node id="17192249" lat="49.6323053" lon="10.9829889" version="3" changeset="1173972" 
	user="GeoGrafiker" uid="69127" timestamp="2009-05-13T11:48:59Z"/>
 <node id="17192250" lat="49.6321964" lon="10.9817792" version="4" changeset="1174159" 
	user="GeoGrafiker" uid="69127" timestamp="2009-05-13T12:08:14Z"/>
 <node id="17193023" lat="49.5980682" lon="11.0037364" version="15" changeset="8691470" 
	user="okilimu" uid="212111" timestamp="2011-07-11T08:26:21Z">
  <tag k="is_in" v="Mittelfranken,Bayern,Bundesrepublik Deutschland,Europe" />
  <tag k="is_in:continent" v="Europe" />
  <tag k="name" v="Erlangen" />
  <tag k="name:de" v="Erlangen" />
  <tag k="name:en" v="Erlangen" />
  <tag k="name:ru" v="Э�\x80ланген" />
  <tag k="name:sr" v="�\x95�\x80ланген" />
  <tag k="openGeoDB:auto_update" v="population,is_in" />
  <tag k="openGeoDB:community_identification_number" v="09562" />
  <tag k="openGeoDB:is_in" v="Mittelfranken,Bayern,Bundesrepublik Deutschland,Europe" />
  <tag k="openGeoDB:is_in_loc_id" v="164" />
  <tag k="opengeodb:lat" v="49.5978347" />
  <tag k="openGeoDB:layer" v="5" />
  <tag k="openGeoDB:license_plate_code" v="ER" />
  <tag k="openGeoDB:loc_id" v="608" />
  <tag k="opengeodb:lon" v="11.0048476" />
  <tag k="openGeoDB:name" v="Erlangen" />
  <tag k="openGeoDB:sort_name" v="ERLANGEN" />
  <tag k="openGeoDB:type" v="Stadt" />
  <tag k="openGeoDB:version" v=" / 2007-12-04 /" />
  <tag k="place" v="city" />
  <tag k="population" v="105554" />
  <tag k="website" v="" />
  <tag k="wikipedia:de" v="Erlangen" />
  1. They have tag tags !!!!
  2. The node tags at the start presumably all refer to the city of Erlangen (since they all have roughly the same lat/lon coordinates), but this can only be inferred by observing that they are placed before a node with actual content naming the city.
  3. The type of a node is hidden in the v attribute of a tag tag's k attribute, iff that attribute has the value place, in the case of Erlangen the v is city. This type seem's not to be checked in any way, I found place attributes with ks of yes, no, Naturflaechendenkmal and various others.
  4. There's no way to automatically verify this mess so why use XML at all?!??
  5. The embedded openGeoDB content repeats values of other attributes. This almost necessarily results in outdated entries, mismatches, and general headaches.
  6. The is_in v value repeats contents of the is_in:continent attribute.
  7. There is a continent Europe in the OSM database, as are the state, county, etc. and their nodes surely have ids. So why not use XML's idref to refer to them?

[/osfail] permanent link

Fri, 13 Jan 2012

MySQL cannot erase data

For eight years it has been known (and Verified) that MySQL does not release diskspace it has claimed in its ibdatax files. The bug leads to full disks and database outages since the only way arround it is to shut down the DB, dump it (consuming even more space) scrubbing the disk and restoring from dump.

This is a veritable showstopper.

[/osfail] permanent link

Mon, 28 Nov 2011


Since NIS has seen its hayday (in the early 90ies), we switched to the highly secure LDAP+Kerberos setup. OpenLDAP is touted as the allround “Solution” to all user account management, sorry, I meant to say account provisioning. After converting our NIS passwd file to thousands of LDIF files we imported them with the obvious three-liner

  for i in *ldif; do \
	ldapadd -v -W -D "cn=admin,dc=our,dc=domain" \
	    -c -H ldapi:/// -f $i

After fiddling with half a dozen files in /etc, the client systems could look up user data on the LDAP server. Our test for that was

  getent passwd aknownuser
  getent passwd anotherknownuser
So it seemed plausible to create a list of all users by
  getent passwd | awk -F: '{print $1}' > allourusers
and install that as a list of valid recipients of e-mails.

This was a grave error, because one of the manymany default settings of OpenLDAP is to return only the first 500 answers to any request. So the list was short by a few hundred accounts.

Considering that LDAP has no concept of a cursor and one cannot ask for the next 500  entries, one can only ask

What the fsck were they thinking???

[/osfail] permanent link

Tue, 25 Jan 2011

Including by explicitly excluding files in Bacula

Bacula's config file format allows the following
FileSet {
	Name = Blah 
	Exclude {
		Options {
			wilddir = /var/tmp*
	File = /var
The Directory /var/tmp from the Exclude section is included in the backup, because to exclude it, the proper config would be
FileSet {
	Name = Blah 
	Exclude {
		Options {
			wilddir = /var/tmp*
		exclude = yes
	File = /var

[/osfail] permanent link