19.5 Wrapping C Environment Calls

Let's move on to a more useful application of C extension modules. The hand-coded C file in Example 19-8 integrates the standard C library's getenv and putenv shell environment variable calls for use in Python scripts.

Example 19-8. PP2E\Integrate\Extend\CEnviron\cenviron.c

/******************************************************************
 * A C extension module for Python, called "cenviron".  Wraps the 
 * C library's getenv/putenv routines for use in Python programs.  
 ******************************************************************/

#include <Python.h>
#include <stdlib.h>
#include <string.h>

/***********************/
/* 1) module functions */
/***********************/
 
static PyObject *                                   /* returns object */
wrap_getenv(PyObject *self, PyObject *args)         /* self not used */
{                                                   /* args from python */
    char *varName, *varValue;
    PyObject *returnObj = NULL;                         /* null=exception */

    if (PyArg_Parse(args, "s", &varName)) {             /* Python -> C */
        varValue = getenv(varName);                     /* call C getenv */
        if (varValue != NULL)
            returnObj = Py_BuildValue("s", varValue);   /* C -> Python */
        else
            PyErr_SetString(PyExc_SystemError, "Error calling getenv");
    }
    return returnObj;
}

static PyObject *
wrap_putenv(PyObject *self, PyObject *args)
{
    char *varName, *varValue, *varAssign;
    PyObject *returnObj = NULL;

    if (PyArg_Parse(args, "(ss)", &varName, &varValue))
    {
        varAssign = malloc(strlen(varName) + strlen(varValue) + 2);
        sprintf(varAssign, "%s=%s", varName, varValue);
        if (putenv(varAssign) == 0) {
            Py_INCREF(Py_None);                   /* C call success */
            returnObj = Py_None;                  /* reference None */
        }
        else
            PyErr_SetString(PyExc_SystemError, "Error calling putenv");
    }
    return returnObj;
}

/**************************/
/* 2) registration table  */
/**************************/

static struct PyMethodDef cenviron_methods[] = {
    {"getenv", wrap_getenv},
    {"putenv", wrap_putenv},        /* method name, address */
    {NULL, NULL}
};

/*************************/
/* 3) module initializer */
/*************************/

void initcenviron(  )                  /* called on first import */
{
    (void) Py_InitModule("cenviron", cenviron_methods);   /* mod name, table */
}

This example is less useful now than it was in the first edition of this book -- as we learned in Part I, not only can you fetch shell environment variables by indexing the os.environ table, but assigning to a key in this table automatically calls C's putenv to export the new setting to the C code layer in the process. That is, os.environ['key'] fetches the value of shell variable 'key', and os.environ['key']=value assigns a variable both in Python and C.

The second action -- pushing assignments out to C -- was added to Python releases after the first edition of this book was published. Besides demonstrating additional extension coding techniques, though, this example still serves a practical purpose: even today, changes made to shell variables by the C code linked in to a Python process are not picked up when you index os.environ in Python code. That is, once your program starts, os.environ reflects only subsequent changes made by Python code.

If you want your Python code to be truly integrated with shell settings made by your C extension modules' code, you still must rely on calls to the C library's environment tools: putenv is available as os.putenv, but getenv is not present in the Python library. This will probably rarely, if ever, be an issue; but this C extension module is not completely without purpose (at least until Guido tightens this up again).^[4]

^[4] This code is also open to customization (e.g., it can limit the set of shell variables read and written by checking names), but you could do the same by wrapping os.environ. In fact, because os.environ is simply a Python UserDict subclass that preloads shell variables on startup, you could almost add the required getenv call to load C layer changes by simply wrapping os.environ accesses in a Python class whose __getitem__ calls gentenv before passing the access off to os.environ. But you still need C's getenv call in the first place, and it's not available in os today.

This cenviron.c C file creates a Python module called cenviron that does a bit more than the last example -- it exports two functions, sets some exception descriptions explicitly, and makes a reference count call for the Python None object (it's not created anew, so we need to add a reference before passing it to Python). As before, to add this code to Python, compile and link into an object file; the Linux makefile in Example 19-9 builds the C source code for dynamic binding.

Example 19-9. PP2E\Integrate\Extend\Cenviron\makefile.cenviron

##################################################################
# Compile cenviron.c into cenviron.so--a shareable object file 
# on Linux, which is loaded dynamically when first imported. 
##################################################################

PY = $(MYPY)

cenviron.so: cenviron.c
    gcc cenviron.c -g -I$(PY)/Include -I$(PY) -fpic -shared -o cenviron.so

clean:
    rm -f *.pyc cenviron.so

To build, type make -f makefile.cenviron at your shell. To run, make sure the .so file is in a directory on Python's module path ("." works too):

[mark@toy ~/.../PP2E/Integrate/Extend/Cenviron]$ python
>>> import cenviron
>>> cenviron.getenv('USER')                # like os.environ[key] but refetched
'mark'
>>> cenviron.putenv('USER', 'gilligan')    # like os.environ[key]=value
>>> cenviron.getenv('USER')                # C sees the changes too
'gilligan'

As before, cenviron is a bona fide Python module object after it is imported, with all the usual attached information:

>>> dir(cenviron)
['__doc__', '__file__', '__name__', 'getenv', 'putenv']
>>> cenviron.__file__
'./cenviron.so'
>>> cenviron.__name__
'cenviron'
>>> cenviron.getenv
<built-in function getenv>
>>> cenviron
<module 'cenviron' from './cenviron.so'>
>>> print cenviron.getenv('HOST'), cenviron.getenv('DISPLAY')
toy :0.0

Here is an example of the problem this module addresses (but you have to pretend that the getenv calls are made by linked-in C code, not Python):

>>> import os
>>> os.environ['USER']                      # initialized from the shell
'skipper'
>>> from cenviron import getenv, putenv     # direct C library call access
>>> getenv('USER')
'skipper'
>>> putenv('USER', 'gilligan')              # changes for C but not Python
>>> getenv('USER')
'gilligan'
>>> os.environ['USER']                      # oops--does not fetch values again
'skipper'

As is, the C extension module exports a function-based interface, but you can wrap its functions in Python code that makes the interface look any way you like. For instance, Example 19-10 makes the functions accessible by dictionary indexing, and integrates with the os.environ object.

Example 19-10. PP2E\Integrate\Extend\Cenviron\envmap.py

import os
from cenviron import getenv, putenv       # get C module's methods

class EnvMapping:                         # wrap in a Python class
    def __setitem__(self, key, value):
        os.environ[key] = value           # on writes: Env[key]=value
        putenv(key, value)                # put in os.environ too

    def __getitem__(self, key):
        value = getenv(key)               # on reads: Env[key]
        os.environ[key] = value           # integrity check
        return value

Env = EnvMapping(  )                        # make one instance   

And Example 19-11 exports the functions as qualified attribute names instead of calls. The point here is that you can graft many different sorts of interface models on top of extension functions by providing Python wrappers (an idea we'll revisit when we meet type wrappers and SWIG shadow classes later in this chapter).

Example 19-11. PP2E\Integrate\Extend\Cenviron\envattr.py

import os
from cenviron import getenv, putenv       # get C module's methods

class EnvWrapper:                         # wrap in a Python class
    def __setattr__(self, name, value):
        os.environ[name] = value          # on writes: Env.name=value
        putenv(name, value)               # put in os.environ too

    def __getattr__(self, name):
        value = getenv(name)              # on reads: Env.name
        os.environ[name] = value          # integrity check
        return value

Env = EnvWrapper(  )                        # make one instance   

19.5.1 But Don't Do That Either -- SWIG

You can manually code extension modules like we just did, but you don't necessarily have to. Because this example really just wraps functions that already exist in standard C libraries, the entire cenviron.c C code file of Example 19-8 can be replaced with a simple SWIG input file that looks like Example 19-12.

Example 19-12. PP2E\Integrate\Extend\Swig\Environ\environ.i

/***************************************************************
 * Swig module description file, to generate all Python wrapper
 * code for C lib getenv/putenv calls: "swig -python environ.i".   
 ***************************************************************/

%module environ 

%{
#include <stdlib.h>
%}

extern char * getenv(const char *varname);
extern int    putenv(const char *assignment);

And you're done. Well, almost; you still need to run this file through SWIG and compile its output. As before, simply add a SWIG step to your makefile, compile its output file into a shareable object, and you're in business. Example 19-13 is a Linux makefile that does the job.

Example 19-13. PP2E\Integrate\Extend\Swig\Environ\makefile.environ-swig

# build environ.so extension from SWIG generated code 

# unless you've run make install
SWIG = ../myswig
PY   = $(MYPY)

environ.so: environ_wrap.c
	gcc environ_wrap.c -g -I$(PY)/Include -I$(PY) -shared -o environ.so

environ_wrap.c: environ.i
	$(SWIG) -python environ.i

clean:
	rm -f *.o *.so core 
force:
	rm -f *.o *.so core environ_wrap.c environ_wrap.doc

When run on environ.i, SWIG generates two files -- environ_wrap.doc (a list of wrapper function descriptions) and environ_wrap.c (the glue code module file). Because the functions being wrapped here live in standard linked-in C libraries, there is nothing to combine with the generated code; this makefile simply runs SWIG and compiles the wrapper file into a C extension module, ready to be imported:

[mark@toy ~/....../Integrate/Extend/Swig/Environ]$ make -f makefile.environ-swig 
../myswig -python environ.i
Generating wrappers for Python
gcc environ_wrap.c -g -I/...  more...  -shared -o environ.so

And now you're really done. The resulting C extension module is linked when imported, and used as before (except that SWIG handled all the gory bits):

[mark@toy ~/....../Integrate/Extend/Swig/Environ]$ python
>>> import environ
>>> environ.getenv('USER')
'mark'
>>> environ.putenv('USER=gilligan')             # use C lib call pattern now
0
>>> environ.getenv('USER')
'gilligan'

>>> dir(environ)
['__doc__', '__file__', '__name__', 'getenv', 'putenv']
>>> environ.__name__, environ.__file__, environ
('environ', './environ.so', <module 'environ' from './environ.so'>)

You could also run SWIG over the C header file where getenv and putenv are defined, but that would result in wrappers for every function in the header file. With the input file coded here, you'll wrap only two library functions.