Bookmarks for 1 July through 19 July

Monday 19 July 2010 - Filed under Bookmarks

• 3 shell scripts: Kill weasel words, avoid the passive, eliminate duplicates – "So, I've decided to replace myself with a shell script. In particular, I've created shell scripts for catching three problems: the passive voice, weasel words, and<br /> lexical illusions."
• [0811.0113] A Bayesian Framework for Opinion Updates – "Opinion Dynamics lacks a theoretical basis. In this article, I propose to use a decision-theoretic framework, based on the updating of subjective probabilities, as that basis. We will see we get a basic tool for a better understanding of the interaction between the agents in Opinion Dynamics problems and for creating new models."
• TexoTela jQuery – newsticker – A fade-in/fade-out newsticker for jQuery.
• jQuery liScroll – a jQuery News Ticker – "liScroll is a jQuery plugin that transforms any given unordered list into a scrolling News Ticker."
• Taguchi methods – Wikipedia, the free encyclopedia – "Taguchi methods are statistical methods developed by Genichi Taguchi to improve the quality of manufactured goods, and more recently also applied to, engineering, biotechnology, marketing and advertising."

Comments Off  ::  Share or discuss  ::  2010-07-19  ::  Stephen

Bookmarks for 8 June through 29 June

Tuesday 29 June 2010 - Filed under Bookmarks

• [1006.5169] Hyperbolic Geometry of Complex Networks – Interesting.
• Learning about Network Theory – A brief introduction, with links to resources.
• NetworkX v1.1 – "NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks."
• [1006.1786] Measuring Meaning on the World-Wide Web – A brief paper exploring word-word correlations on the Web, with some interesting specific examples.
• [1006.1231] On the Insertion Time of Cuckoo Hashing – "Cuckoo hashing is an efficient technique for creating large hash tables with high space utilization and guaranteed constant access times. There, each item can be placed in a location given by any one out of k different hash functions."

Comments Off  ::  Share or discuss  ::  2010-06-29  ::  Stephen

adirent.[ch]: Adding d_type to struct dirent on OpenSolaris

Sunday 13 June 2010 - Filed under Software

An occasional porting problem you may encounter when compiling programs for OpenSolaris is the absence of d_type in the directory entry structure returned by readdir(3C). I hit this issue when experimenting with mu as a search solution for my accumulated email.

A trivial example of the failure you might see would be caused by the following program:

#include <sys/types.h>
#include <dirent.h>
#include <err.h>
#include <stdio.h>

int
main(int argc, char *argv[])
{
        DIR *d;
        struct dirent *e;

        if ((d = opendir("/")) == NULL)
                err(1, "opendir failed");

        for (e = readdir(d); e != NULL; e = readdir(d)) {
                    if (e->d_type != DT_UNKNOWN)
                        (void) printf("recognized filetype for '%s'\n",
                            e->d_name);
        }

        (void) closedir(d);

        return (0);
 }

When we attempt to compile this program with gcc, we get something like

$ gcc a.c
a.c: In function `main':
a.c:16: error: structure has no member named `d_type'
a.c:16: error: `DT_UNKNOWN' undeclared (first use in this function)
a.c:16: error: (Each undeclared identifier is reported only once
a.c:16: error: for each function it appears in.)

Studio cc will give similar output:

$ /opt/SunStudioExpress/bin/cc a.c
"a.c", line 16: undefined struct/union member: d_type
"a.c", line 16: undefined symbol: DT_UNKNOWN
cc: acomp failed for a.c

The addition of d_type to struct dirent came first for the BSD Unixes and was later added to Linux. Because it’s not easy to add members to well-known structures and preserve binary compatibility, OpenSolaris and Solaris lack this field, as well as the DT_* constant definitions. (If d_type were to become part of the Unix standards, Solaris would likely have to introduce a second family of opendir()/readdir()/closedir() functions and a second version of the structure, similar to how large files were introduced for 32-bit programs.)

Because we fail at compilation time, our workaround has to modify either the program’s source code or its build environment. (Preloading is too late.) It’s probably possible to combine a few definitions and a shared object that we include via LD_PRELOAD but it seems easier to just provide a C wrapper around readdir(3C) and an alternate struct dirent. We develop this approach in the next section.

DIRENT and READDIR

The approach we take is

1. Introduce DIRENT and READDIR via adirent.h.
2. Change the source program such that each call to readdir() is replaced by READDIR() and each use of struct dirent is replaced by DIRENT. In each file so modified, add a #include <adirent.h>.
3. Compile adirent.c via gcc -I. -O2 -c adirent.c or equivalent.
4. Add adirent.o to the link line for each binary that includes one of the files modified in step 2.

If we apply these steps to our example above, we get

#include <sys/types.h>
#include <adirent.h>
#include <err.h>
#include <stdio.h>

int
main(int argc, char *argv[])
{
        DIR *d;
        DIRENT *e;

        if ((d = opendir("/")) == NULL)
                err(1, "opendir failed");

        for (e = READDIR(d); e != NULL; e = READDIR(d)) {
                    if (e->d_type != DT_UNKNOWN)
                        (void) printf("recognized filetype for '%s'\n",
                            e->d_name);
        }

        (void) closedir(d);

        return (0);
 }

with the result that compilation and execution now work

$ gcc -O2 -I. -c adirent.c
$ gcc -I. a.c adirent.o
$ ./a.out

This shim function and definitions should be sufficient for most ports around this incompatibility, but there are some additional comments worth making.

Performance. Because many programs expect d_type to be one of DT_REG or DT_DIR to save on a stat(2) call, this shim will force those programs into an alleged “slow” path. The actual impact of returning DT_UNKNOWN on every call will be program- and situation-dependent; it didn’t seem to affect my mail indexing.

Multithreaded programs. The current implementation does not protect the static structure defined in adirent.c. Programs with multiple threads performing readdir(3C) calls through READDIR() will get unexpected results. It should be relatively straightforward to dynamically allocate one struct adirent for each thread coming through READDIR() for the first time.

Downloads

I suppose these should be in a repository on Bitbucket or GitHub. For now, they’re just simple downloads:

• adirent.c
• adirent.h

Acknowledgments

I discussed this problem with Dan, who in particular noted that DT_UNKNOWN was always a legal return value for d_type. Bart looked over my shoulder and spied at least one error during the debugging phase.

Comments Off  ::  Share or discuss  ::  2010-06-13  ::  Stephen

A cruise for World Oceans Day

Saturday 12 June 2010 - Filed under Family + Peninsula + Photo

We went on the Marine Science Institute‘s cruise today, which was in part to celebrate World Oceans Day 2010. Part of the cruise involved sampling the bay floor and a short trawl with a net. (All animals caught are returned to the bay later in the day.) Ben got to hold a juvenile leopard shark.

Comments Off  ::  Share or discuss  ::  2010-06-12  ::  Stephen

Bookmarks for 18 May through 2 June

Wednesday 2 June 2010 - Filed under Bookmarks

• [1006.0200] The 1958 Pekeris-Accad-WEIZAC Ground-Breaking Collaboration that computed Ground States of Two-Electron Atoms (and its 2010 Redux) – Koutschan and Zeilberger examine and "re-enact" Pekeris's 1958 calculation of the ground state of the two-electron atom.
• Dictionary of Algorithms and Data Structures – "This is a dictionary of algorithms, algorithmic techniques, data structures, archetypal problems, and related definitions." Hosted at NIST.
• pkg(5) on ohloh.net – pkg(5)'s repository is now mirrored at genunix.org, so we have ohloh's code and commit analysis. I'll never catch any of those guys.
• Towards Enterprise-Class Compatibility for Apache Hadoop (Hadoop and Distributed Computing at Yahoo!) – The Yahoo! Hadoop team has classified the Hadoop interfaces, using a taxonomy derived from the OpenSolaris ARC interface taxonomy.
• Eric Schrock, Debugging AJAX in production – ACM Queue – I missed this article last year, but Eric's written a nice essay with some example code for improving the debuggability of AJAX applications.

Comments Off  ::  Share or discuss  ::  2010-06-02  ::  Stephen

Pipes arrive

Monday 17 May 2010 - Filed under Observations + Peninsula + Photo

This morning, the pipe segments began to arrive for the section of Bay Division Pipeline No. 5 nearest us. The pipes are a distinct dark blue-green, with the ends capped with paper.

Comments Off  ::  Share or discuss  ::  2010-05-17  ::  Stephen

Bookmarks for 3 May through 13 May

Thursday 13 May 2010 - Filed under Bookmarks

• Social Page Evaluator – "Our solution is designed to provide an assessment of a brand's Facebook Page value and the effectiveness of its social media efforts."
• SFWATER.ORG : BDPL Reliability Upgrade – Pipeline Peninsula – Project status page for the BDPL5 Peninsula project.
• Water System Improvement Program BDPL5Peninsula – Blog for the Bay Division Pipeline No. 5 Peninsula construction project.
• Shoot, edit, and manage photos in Android – PC World – A good intro to the Camera and Gallery applications for Android.
• A Professional Photo Workflow for OpenSolaris « Unwinding the Stack – Comments on Argyll, LensFun, and UFRaw, all available on OpenSolaris and all available via pkg(5) from http://pkg.thestaticvoid.com:10000

Comments Off  ::  Share or discuss  ::  2010-05-13  ::  Stephen

Large machine, trench work

Saturday 8 May 2010 - Filed under Observations + Peninsula + Photo

They assembled this machine this week, but haven’t fired it up. The pipeline runs right behind our property, so I expect we’ll hear the crunching of treads and a large motor (or more than one) fire up early in the morning soon.

When you’re digging a miles-long trench, there’s a clear axis of work. And it looks like they’ll be removing the road surface soon.

Comments Off  ::  Share or discuss  ::  2010-05-08  ::  Stephen

Reasonable photos from 2009

Wednesday 5 May 2010 - Filed under General + Observations + Photo

I’ve started uploading a subset of my photos to Flickr.  Here are four from 2009 I’m reasonably happy with, with some commentary.

We took a short trip north to the coast in the spring, camping a few nights at the Bodega Dunes site in Sonoma Coast SP. This bird was interested in our activities, although not as aggressive as some distant cousins at San Simeon.

NASA Ames had an event last year, with much model rocketry on the airfield. This model Saturn V was one of the most impressive rockets fired, and I was happy to catch it as it took off.

The dock at the cottage. Weathered, if not gnarly.

I was amused by the modification to this sign. (O.P.P. is short for Ontario Provincial Police; C.P.O. for Community Policing Office; C-3PO for a particular protocol droid.)

Comments Off  ::  Share or discuss  ::  2010-05-05  ::  Stephen

Tent interior at Peter Pan 360

Wednesday 5 May 2010 - Filed under City + Observations + Photo

We went up to San Francisco to see the Peter Pan show at Ferry Park today. The kids were very excited by the human-powered ticking crocodile. There’s a large concession tent for use before the show and at intermission; this shot is of one of its peaks.

Comments Off  ::  Share or discuss  ::  2010-05-05  ::  Stephen