gcc-python-plugin October 2011

gcc-python-plugin@lists.stg.fedorahosted.org

5 participants
12 discussions

[patch] bug in gcc_python_closure_new_generic
by Tom Tromey 31 Oct '11

31 Oct '11

Consider this plugin: class Whatever: def doit(self, optpass, fun, *args, **kwargs): print "hi bob" def main(**kwargs): w = Whatever() gcc.register_callback(gcc.PLUGIN_PASS_EXECUTION, w.doit) When I try to run this (invoking main...), I get an ICE: *** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins. Event | Plugins PLUGIN_FINISH | python PLUGIN_PASS_EXECUTION | python ../../archer/gdb/python/py-type.c: In function ‘gdbpy_initialize_types’: ../../archer/gdb/python/py-type.c:1378:1: internal compiler error: Segmentation fault The bug is a missing incref in gcc-python-closure.c. Patch appended. Tom diff --git a/gcc-python-closure.c b/gcc-python-closure.c index 5c9b8ae..f238914 100644 --- a/gcc-python-closure.c +++ b/gcc-python-closure.c @@ -39,6 +39,7 @@ gcc_python_closure_new_generic(PyObject *callback, PyObject *extraargs, PyObject } closure->callback = callback; + Py_INCREF(callback); // FIXME: we may want to pass in the event enum as well as the user-supplied extraargs

2 1

Tracking the range of possible values for an r-value in cpychecker
by David Malcolm 28 Oct '11

28 Oct '11

I've slightly reworked how cpychecker tracks unknown values in http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commitdiff;h=207… For integer types, it now uses a new WithinRange abstract value, describing the closed interval of possible values i.e.: minvalue <= x <= maxvalue For example, given this test case: extern void __cpychecker_dump(int); char array[12] = {2, 2, 2, 2, 8, 1, 8, 1, 8, 2, 8, 2}; char test(int i) { __cpychecker_dump(i); /* This ought to constrain i to within [0-255]: */ i = i & 0xff; __cpychecker_dump(i); /* and this ought to constrain it to within [0-7]: */ i = i >> 5; __cpychecker_dump(i); /* and hence this read is within the array bounds: */ return array[i]; } we can see the values the analyzer knows about, using the "magic" function __cpychecker_dump(): __dump((int)val [-0x80000000 <= val <= 0x7fffffff]) __dump((int)val [0 <= val <= 255]) __dump((int)val [0 <= val <= 7]) i.e. it starts by treating "i" as any possible int, but the mask against 0xff gives a new WithinRange(0, 255), and the right-shift constrains it again. In theory we can use this to do bounds-checking of array/buffer accesses. I'm still working through the errors we were discussing in the "python exceptions from refcount checker" thread, so I'm holding off on implementing that yet. I ran into this when compiling a fragment of code similar to the above in gdb; the difference there is that the array was a local, initialized within the function scope. cpychecker was complaining that the array access was a read of an uninitialized value, since it wasn't able to prove that "i" was within the bounds. Unfortunately, even with this fix, it still complains, since it isn't smart enough to know that all possible indices have an initialized value, and picks the "default" value for the array, which is "uninitialized". Hopefully I can fix that case too. Dave

2 3

Custom attributes
by David Malcolm 27 Oct '11

27 Oct '11

I've just committed initial support for creating custom compilation attributes from Python, as: http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commitdiff;h=a9a… So it's now possible to write declarations like these: void some_function(void) __attribute__(claims_mutex("io")); void some_other_function(void) __attribute__(releases_mutex("io")); and register those new attributes from Python code, and have a code analysis script use them to ensure some property (e.g. that the resource claim/release usage is properly balanced). See: http://readthedocs.org/docs/gcc-python-plugin/en/latest/attributes.html Next step will be to implement some attributes in the cpychecker for marking up: * functions that steal references to their PyObject* arguments * functions that return borrowed references * functions for which a negative return code indicates that an exception has been set Hopefully those will cover the bulk of the false-positives that we're still seeing on compiling gdb's python code with cpychecker. Dave

3 10

Revamped documentation for gcc-with-cpychecker
by David Malcolm 26 Oct '11

26 Oct '11

I've fallen into bad habits, and haven't been updating the documentation to the cpychecker code. Sorry about that. I've spent a few hours fixing that, and you can now see relatively up-to-date docs for this part of the project here: http://readthedocs.org/docs/gcc-python-plugin/en/latest/cpychecker.html I'll try to be better from now on about updating the documentation as this part of the code changes. Dave

1 0

Emulating gcc-xml?
by Alek Paunov 25 Oct '11

25 Oct '11

Hi, Regarding various possible use-cases, Is it currently possible to emulate/enhance gcc-xml export using a plugin script? I'd tried, if the data is accessible. Thank you, Alek P.S. Perhaps, it is worth to replicate this plugin too (I love sqlite :-)): http://code.google.com/p/gccsymdb/

2 1

Obtaining absolute paths when obtaining locations
by satya komaragiri 25 Oct '11

25 Oct '11

Hi, I am trying to use gcc-python-plugin to obtain caller-callee information in JSON format and I need to obtain the location as well. I am using callgraphnode.decl.location and edge.caller.decl.location for this. When the file is located in the current directory , I get relative paths. Is there any way to always obtain absolute paths? Regards, Satya

3 2

gcc-with-cpychecker now performs refcount checking
by David Malcolm 21 Oct '11

21 Oct '11

As of commit 7cd150fe609b606f259af18fc98d34b4fbde20a7, the gcc-with-cpychecker harness now invokes the reference-count checker, removing the need to hack up libcpychecker/__init__.py to do this. So now you can invoke the checker on arbitrary CPython extension code (e.g. gdb) with something like: make CC=/path/to/built/plugin/gcc-with-cpychecker without having to hack up the checker's code: you'll get a GCC with the extra "cpychecker" compilation pass, which runs these tests: * format checking of Py_ArgParse* and Py_BuildValue calls * reference count analysis Having said that, expect to run into tracebacks and other bugs with the checker if you do (including incorrect reports - so proceed with caution!). I'm still slowly working through gdb and the plugin's code with the checker, fixing issues as I see them. Adding some kind of custom attributes to mark functions that steal references (and those that return borrowed references) is probably next. Dave

1 0

Success stories
by David Malcolm 20 Oct '11

20 Oct '11

I've gone ahead and created a "Success Stories" page within the plugin's documentation (docs/success.rst). I took the liberty of adding links to the various bugs we've been finding in gdb - I hope that's OK - I don't want to make it appear that I'm picking on gdb (sorry!), but I did want to capture the benefit of the plugin in one place. We can reword things, of course. You can see the HTML form of the page here: http://readthedocs.org/docs/gcc-python-plugin/en/latest/success.html [I hope that tracking all of this manually will eventually become unmanageable, which would be, as they say, a good problem to have] I'll expand that page to add the various bugs that the plugin shows up in itself, though I'm not sure that really counts as a "success" :) Has anyone else found the plugin useful so far? Dave (goes back to fixing bugs in cpychecker that compiling gdb shows up)

2 2

python exceptions from refcount checker
by Tom Tromey 17 Oct '11

17 Oct '11

I tried out the refcount checker on gdb today. I'm using git head as of a few minutes ago, with a newly-updated F15 system gcc. I hacked cpychecker.py: diff --git a/cpychecker.py b/cpychecker.py index e120c53..4c24e3f 100644 --- a/cpychecker.py +++ b/cpychecker.py @@ -16,4 +16,4 @@ # <http://www.gnu.org/licenses/>. from libcpychecker import main -main() +main(verify_refcounting = True) (Note that this patch causes test suite failures -- but I think that can't be very important.) I wrote this little script to run 'make' with the right options: #!/bin/sh here=/home/tromey/Space/Trunk/gcc-python-plugin/ make CFLAGS="-gdwarf-4 -g3 -fplugin=$here/python.so -fplugin-arg-python-script=$here/cpychecker.py" "$@" Then I ran it on gdb like: /tmp/Makeit -k -C ~/gnu/archer/build/gdb -j2 I get many failures like this: ../../archer/gdb/armbsd-tdep.c: In function ‘armbsd_regset_from_core_section’: ../../archer/gdb/armbsd-tdep.c:127:3: error: Unhandled Python exception raised calling 'execute' method Traceback (most recent call last): File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/__init__.py", line 49, in execute self.show_possible_null_derefs) File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/refcounts.py", line 2020, in check_refcounts limits=Limits(maxtrans=1024)) File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/absinterp.py", line 1821, in iter_traces f_new = facet_cls(curstate, fun=fun) File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/refcounts.py", line 224, in __init__ self.exception_rvalue = ConcreteValue(get_PyObjectPtr(), File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/types.py", line 60, in get_PyObjectPtr return get_global_typedef('PyObject').pointer AttributeError: 'NoneType' object has no attribute 'pointer' ../../archer/gdb/armbsd-tdep.c: In function ‘armbsd_regset_from_core_section’: ../../archer/gdb/armbsd-tdep.c:127:3: error: Unhandled Python exception raised calling 'execute' method Traceback (most recent call last): File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/__init__.py", line 49, in execute self.show_possible_null_derefs) File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/refcounts.py", line 2020, in check_refcounts limits=Limits(maxtrans=1024)) File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/absinterp.py", line 1821, in iter_traces f_new = facet_cls(curstate, fun=fun) File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/refcounts.py", line 224, in __init__ self.exception_rvalue = ConcreteValue(get_PyObjectPtr(), File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/types.py", line 60, in get_PyObjectPtr return get_global_typedef('PyObject').pointer AttributeError: 'NoneType' object has no attribute 'pointer' The problem here is that some files, like this one, do not use Python at all, and don't include the Python headers. In this situation the plugin dies when it fails to look up a Python type. Tom

2 13

FYI: const fix in py-type.c
by Tom Tromey 13 Oct '11

13 Oct '11

I'm checking this in. David Malcolm's GCC plugin found a bug in py-type.c -- we were using PyArg_ParseTuple's "s" format but passing in a char*, not a const char*. The fix is appended. Thanks again to David for this plugin. Tom 2011-10-13 Tom Tromey <tromey(a)redhat.com> * python/py-type.c (typy_has_key): Make 'field' const. Index: python/py-type.c =================================================================== RCS file: /cvs/src/src/gdb/python/py-type.c,v retrieving revision 1.24 diff -u -r1.24 py-type.c --- python/py-type.c 9 Oct 2011 19:21:38 -0000 1.24 +++ python/py-type.c 13 Oct 2011 13:48:41 -0000 @@ -1176,7 +1176,7 @@ typy_has_key (PyObject *self, PyObject *args) { struct type *type = ((type_object *) self)->type; - char *field; + const char *field; int i; if (!PyArg_ParseTuple (args, "s", &field))

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

gcc-python-plugin October 2011