If you do a lot of work with Python’s standard C implementation, CPython, there’s a chance that you’ll run into issues that will actually crash python.exe. If you’re writing extensions for python in C/C++ (.pyd files), it’s likely that some of those crashes are the result of your code. Naturally, if your C++ caused python.exe to crash, you’re going to attach a debugger and take a look.

If the problem is in your code, often it’s simple to see it – just look at your argument values and data structures. But what if the problem exists outside your code, and somewhere in python? All you have in the debugger are a bunch of PyObject * values; Visual C++ or WinDBG won’t do much to help you work with them. Faced with this, your best options are usually to add debugging code to your python – print statements, assertions, etc. Eventually, you might be able to puzzle out the problem and a solution.

But as it turns out, PyObject doesn’t have to be a black box. If you’re using Visual C++, you have all the tools you need to dig around inside of CPython – even if you just caught an unhandled exception.

For this to work, you’ll need a recent version of Visual C++, along with debug symbols for python25.dll. You also need debug symbols for your C/C++ extension (the .pyd). Once you’ve got all this, attach the visual C++ debugger to your application, and either set a breakpoint in your code, or wait for an unhandled exception.

Once you’re paused in the debugger, inside your code, you’re ready to get to work. Let’s use a hypothetical example. We’ve got a simple object, written in C, with a method exposed to python called ‘print’. The function we wrote in C to handle the method is called MyPrint:

void MyPrint(PyObject * self, PyObject * args) {
    const char *text;

    if (!PyArg_ParseTuple(args, "s", &text))
        return NULL;

    int result = printf("%s", text);

    return Py_BuildValue("i", result);
}

The function is simple – we take a single argument from the python dict (args), convert it to a string, and call printf to print it to standard output. Then we return printf’s return value (the number of characters written) as a python int.

So, let’s suppose that MyPrint crashes. We’re now sitting in the debugger, and staring at a couple opaque PyObject * pointers. If we’re lucky, we got past the ParseTuple call and the ‘text’ pointer is valid too. Unfortunately, the debugger isn’t going to be very helpful. If you look at ’self’ and ‘args’ in the Locals or Watch windows, you’re going to see something like this:

args     0x1179a650 {ob_refcnt=3 ob_type=0x023706a0}

We can see a reference count, and a pointer to the object’s type structure, but that’s about it. There are a few useful things in the type structure, like the name of the type, but that’s it. The same goes for ‘args’ – we can’t see the values inside it. What we really want is to be able to look at MyPrint’s arguments, to see what was passed in.

If you look closely, you’ll notice that the type structure has a field named ‘tp_repr’. If you dig into the CPython documentation, you’ll realize that tp_repr is a pointer to the repr() function for that object. In python, repr() converts any given object into a string, usable for debugging purposes.

Go ahead and open up Visual C++’s Immediate window. You’ll find it under the Debug menu’s Windows submenu. After it’s open, type this in:

args->ob_type->tp_repr(args)

If everything’s in good shape, you’ll see output like this appear in the Immediate window:

0x1196fd88 { ob_refcnt=2 ob_type=0x02370600 }

You just called repr() on your argument list, and it returned a Python string object. Now we’re in a bit of a tough situation – we still don’t have any way to look at our results. All we have is a Python object. We do, however, know that it’s a string, and as it turns out, in CPython, this means that the text of the string is directly adjacent to the PyObject structure. Knowing that, we can read the contents of memory to see the string.

Let’s take the PyObject * we got from repr(), and have visual studio show us the contents of the memory at that address. Type this into the immediate window:

0x1196fd88,ma

The ‘,ma‘ is a hint to the Visual Studio Expression Evaluator (used by the Watch and Immediate windows) that it should treat the specified value as a Memory address pointing to ASCII text. It’ll spit out something like this:

0x1196fd88  ..... ..6...........(<MyClass object at 0xbaadf00d>)............

The debugger wrote out the first 64 bytes of memory at that location. The first few bytes are the contents of the PyObject structure, and immediately following them, the contents of the string. From the result of repr(), we can now see that an instance of MyClass was passed to MyPrint, instead of a string. And from the address shown, it looks like it’s a bad pointer.

This is already useful debugging information we might not have had access to otherwise. But we can go further!

Let’s suppose that the MyClass object that was passed to us is actually valid. In that case, you’ll see a repr() that looks innocuous, and have an idea of what the contents of your argument list are. But if your arguments are all objects, repr() won’t tell you that much about them.

If MyClass looks like this:

class MyClass:
  def __init__(self, number, name):
    self.__number = number
    self.name = name

Calling repr() on it won’t tell us the things we actually care about – its name and number. We could add a repr() function to MyClass to find this out, if we wanted. But if you’re trying to debug a rare crash that’s hard to reproduce, restarting your app to add a repr() function is a dangerous step to take – you might not see the crash again for hours, days, or weeks.

So, let’s find out the name and number of the MyClass instance that was passed in.

The first step is to pull the instance out of our argument list. If we wanted, we could just grab the address straight out of the repr() – but let’s be SLIGHTLY less evil and do it using the CPython API. Type this into the immediate window:

{,,python25.dll}PySequence_GetItem(args, 0)

If everything went well, you’ll see output like this:

0xbaadf00d { ob_refcnt=4 ob_type=0x023791010 }

The expression we just evaluated bears a little explanation: The ‘{,,python25.dll}‘ part of the expression tells the debugger that you want to resolve the following symbol using the module ‘python25.dll‘. Otherwise, the debugger might not know the identity of PySequence_GetItem, depending on the quality of your loaded symbols. Don’t ask me why the two commas are there, I have no idea.

So, now that we’ve successfully called GetItem on our argument list, we have the PyObject * for our MyClass instance. If things worked correctly, it will match what you saw in the result of repr().

Now that we have a MyClass instance, we want to get the name and number. Let’s get the value of the name attribute:

((PyObject *)0xbaadf00d)->ob_type->tp_getattro( (PyObject *)0xbaadf00d, {,,python25.dll}PyString_FromString("name") )

There’s a lot going on here, so let’s take it apart. First, we tell the debugger to treat the pointer we were given as a PyObject *:

((PyObject *)0xbaadf00d)

Then we invoke its ‘tp_getattro’ handler, which is the generic getattr() handler for the object. Note that we have to pass in the pointer again, since it takes self as a parameter:

->ob_type->tp_getattro( (PyObject *)0xbaadf00d,

Then finally, we pass in the name of the attribute we wish to retrieve, as a PyObject. To do this, we create a PyString, using the CPython API:

{,,python25.dll}PyString_FromString("name") )

As before, we had to help the debugger find the name.

The getattr call should succeed, and you should get a result:

0xabcd0123 { ob_refcnt=7 ob_type=0x023792222 }

Using the tricks we’ve already learned, we can peek at the name:

0xabcd0123, ma

0xabcd0123  ..... ..6...........Steve..............................

Great. Given this, we may already have enough information to hunt through our source code, looking for the code that constructs Steve. But let’s pretend we need the number:

((PyObject *)0xbaadf00d)->ob_type->tp_getattro( (PyObject *)0xbaadf00d, {,,python25.dll}PyString_FromString("number") )

0x00000000

Hm. That didn’t work. If we remember carefully, MyClass stores the number in an attribute named ‘__number’, not ‘number’. That’s easy to fix, right?

((PyObject *)0xbaadf00d)->ob_type->tp_getattro( (PyObject *)0xbaadf00d, {,,python25.dll}PyString_FromString("__number") )

0x00000000

No dice. What’s the problem? If you read the Python documentation, you’ll find that a double underscore at the beginning of an attribute’s name gets treated specially by the language, and ‘mangled’ in order to protect it from external access. As a result, when we wrote:

    self.__number = number

What actually got run was:

    self._MyClass__number = number

This results in __number being accessible from inside MyClass, but not outside. Knowing this, we can fix our getattr call:

((PyObject *)0xbaadf00d)->ob_type->tp_getattro( (PyObject *)0xbaadf00d, {,,python25.dll}PyString_FromString("_MyClass__number") )

0xdcba3210 { ob_refcnt=5 ob_type=0x023794444 }

Since number isn’t likely to be a string, let’s call repr and see what it is, and for convenience, display the result as a string directly:

((PyObject *)0xdcba3210)->ob_type->tp_repr( (PyObject *)0xdcba3210),ma

0xddddaaaa  ..... ..8...........42..............................

Now we know that MyPrint was called with an instance of MyClass, that was constructed with the name ‘Steve’ and the number ‘42′, and we never had to leave the debugger. Our app is still running, so we can keep putting the debugger to use to investigate. If MyClass held references to other objects, we could follow those references until we found the information we were looking for.

If you find yourself working with large python strings in the debugger (for example, the repr() of a complex object), you can use this snippet to see the entire contents of an object’s repr(), as a C string:

{,,python25.dll}PyString_AsString(obj->ob_type->tp_repr(obj))

Hopefully the techniques I’ve shown you will prove useful the next time you’re trying to figure out a nasty python crash!

Tags: C++, cpython, debugging, python