gcc-python-plugin November 2011

gcc-python-plugin@lists.stg.fedorahosted.org

3 participants
7 discussions

Newbie questions
by Stephan Bergmann 02 Dec '11

02 Dec '11

Hi all, Playing around with the plugin (in the hope of eventually using it to find out interesting things in the LibreOffice code base), I'm not sure I'm trying to use it in a sensible manner. Maybe somebody on this list can set me on track? As a first experiment, I wanted to do the following: LO has a string class rtl::OUString with member functions getLength and isEmpty. Now, I wanted to find places in the code that call getLength and check the result against null, instead of calling isEmpty directly. My first doubt is whether doing that at Gimple-level is practical at all (and whether the plugin would also support doing it at another, more appropriate level that is closer to the original AST)? I figure I need to find GimpleCall statements that call rtl::OUString::getLength and trace their lhs variables (potentially through further GimpleAssign statements) to GimpleCond statements. Something like class TryOut(gcc.GimplePass): def execute(self, fun): for bb in fun.cfg.basic_blocks: if bb.gimple: for stmt in bb.gimple: print ' stmt:', stmt if isinstance(stmt, gcc.GimpleCall): name = stmt.fndecl.name # ... elif isinstance(stmt, gcc.GimpleAssign): # ... elif isinstance(stmt, gcc.GimpleCond): # ... But the first obstacle is already how to determine the full name of the called function, as stmt.fndecl.name would only be "getLength", without the class and namespace information. Any pointers appreciated, Stephan

3 7

ANN: gcc-python-plugin 0.7
by David Malcolm 29 Nov '11

29 Nov '11

gcc-python-plugin is a plugin for GCC 4.6 onwards which embeds the CPython interpreter within GCC, allowing you to write new compiler warnings in Python, generate code visualizations, etc. It ships with "cpychecker", which implements static analysis passes for GCC aimed at finding bugs in CPython extensions. In particular, it can automatically detect reference-counting errors: http://gcc-python-plugin.readthedocs.org/en/latest/cpychecker.html Highlights of this release: * You can now create custom GCC attributes from Python, allowing you to add custom high-level annotations to a C API, and to write Python scripts that will verify these properties. * The "cpychecker" code has too many improvements to list here, and has now detected many reference-counting bugs in real-world CPython extension code. Some success stories can be seen at: http://gcc-python-plugin.readthedocs.org/en/latest/success.html The usability and signal:noise ratio is greatly improved over previous releases. It can now emit HTML reports showing the path of execution through a function that triggers a particular error. For more information you can read detailed notes on this release at: http://readthedocs.org/docs/gcc-python-plugin/en/latest/0.7.html Tarball releases are available at: https://fedorahosted.org/releases/g/c/gcc-python-plugin/ Prebuilt-documentation can be seen at: http://gcc-python-plugin.readthedocs.org/en/latest/index.html The project's homepage is: https://fedorahosted.org/gcc-python-plugin/ The plugin and checker are Free Software, licensed under the GPLv3 or later. Thanks to Red Hat for funding their development. If you're using Fedora, pre-built RPMs of the plugin should be hitting the repositories shortly. Enjoy! Dave Malcolm

1 0

Deduplication of error reports in cpychecker
by David Malcolm 28 Nov '11

28 Nov '11

cpychecker tries to track every possible trace of execution through a function, and so it can sometimes spam you with large numbers of seemingly duplicate error reports for a single underlying root cause. So I've implemented some simple deduplication of error reports within the cpychecker (as of a90611cfbd57a75c80351f3250e2388255e3943b in git). The new code gathers up error reports that are sufficiently similar to each other ("equivalence classes" if you want some mathematical jargon). For now the criteria are a little simplistic: two reports are equivalent if they are for the same function, at the same location, and with the same summary text. It only issues a detailed report for the first error it sees in each partition, together with a note on how many other similar errors it saw. This seems to improve the signal:noise ratio of cpychecker: it successfully removes a genuine duplicate within the test suite [1] and removes quite a few genuine duplicates when recompiling the plugin using gcc-with-cpychecker, making that output much easier to read. But if it seems to be collapsing reports together too aggressively we could tune it a bit, I guess (e.g. by adding some hint fields to error reports for the deduplication code to compare). Dave [1] tests/cpychecker/refcounts/PyDict_SetItem/incorrect

1 0

Failing "make" self-tests
by Stephan Bergmann 28 Nov '11

28 Nov '11

Hi all, With a recent gcc-python-plugin.git repo (as of last Friday), doing "make" failed a lot of the included self-tests (see the attached log.txt). The resulting plugin appears to work just fine, however. This was done on a Fedora-16 x86_64 machine with gcc-4.6.2-1.fc16.x86_64 gcc-plugin-devel-4.6.2-1.fc16.x86_64 python-2.7.2-5.2.fc16.x86_64 python-devel-2.7.2-5.2.fc16.x86_64 python3-3.2.1-1.fc16.x86_64 (Maybe the existence of python3 is a problem? I'm no big Python user, don't know if those two interfere with each other in any way?) Stephan

2 2

Associating run-time and compile-time type information
by David Malcolm 23 Nov '11

23 Nov '11

I added a new custom compiler attribute to the cpychecker, which allows you to associate a PyTypeObject instance with a C typedef: This means that the cpychecker can now tie together run-time type information with C's compile-time type information: /* Define some PyObject subclass, as both a struct and a typedef */ struct OurObjectStruct { PyObject_HEAD /* other fields */ }; typedef struct OurObjectStruct OurExtensionObject; /* Declare the PyTypeObject (run-time type info), using the custom attribute to associate it with the typedef above: */ extern PyTypeObject UserDefinedExtension_Type CPYCHECKER_TYPE_OBJECT_FOR_TYPEDEF("OurExtensionObject"); The checker uses this so far in just one specific place, but it could be used for various other verifications (e.g. to check that casts are sane). This is in http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commit;h=ff3eb4a… See: http://gcc-python-plugin.readthedocs.org/en/latest/cpychecker.html#associat… Naturally the plugin itself now uses this attribute, for the case where the checker is verifying the plugin; doing so fixes some false positives. This is in git in 223bccfc32603be851e0d9b62aee8fd999c329eb. Dave

1 0

gcc plugin for checking gdb's exception handlers
by Tom Tromey 23 Nov '11

23 Nov '11

I wrote a GCC plugin, using David Malcolm's excellent Python plugin, to check for proper use of gdb exceptions in the Python code. I added new 'gdb_throw' and 'gdb_nothrow' attributes and marked a couple spots in the sources. The plugin examines all the calls in a file and writes out a sort of call graph as Python code. Then, after rebuilding GDB with the plugin in place, there is a second program that reads all the data, propagates the "can-throw" property through the call graph, and reports errors whenever a nothrow function is discovered to throw. It reports the path from the function to the throw, like this: ../../archer/gdb/python/python.c:481: error: function gdbpy_decode_line is marked nothrow but can throw ../../archer/gdb/python/python.c:497: info: via call to get_current_arch ../../archer/gdb/arch-utils.c:758: info: via call to get_selected_frame ../../archer/gdb/frame.c:1360: info: via call to error ../../archer/gdb/utils.c:830: info: via call to throw_verror I also had it mark cleanup functions as 'nothrow', since I thought that is either the rule or what the rule should be. This showed a number of errors, enough that I am not sure what I want to do about it yet. I'm attaching the plugin and the analysis program. If you run it yourself, please note that it still emits some false reports. You must examine each one, especially the ones arising from indirect function calls, to see whether it is real. I plan to run it periodically to check for new bugs. To that end I will probably put in the needed gdb patch, also attached. I filed a bug for all the reports in the Python code. Tom

2 1

[patch] bug in gcc_python_closure_new_generic
by Tom Tromey 31 Oct '11

31 Oct '11

Consider this plugin: class Whatever: def doit(self, optpass, fun, *args, **kwargs): print "hi bob" def main(**kwargs): w = Whatever() gcc.register_callback(gcc.PLUGIN_PASS_EXECUTION, w.doit) When I try to run this (invoking main...), I get an ICE: *** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins. Event | Plugins PLUGIN_FINISH | python PLUGIN_PASS_EXECUTION | python ../../archer/gdb/python/py-type.c: In function ‘gdbpy_initialize_types’: ../../archer/gdb/python/py-type.c:1378:1: internal compiler error: Segmentation fault The bug is a missing incref in gcc-python-closure.c. Patch appended. Tom diff --git a/gcc-python-closure.c b/gcc-python-closure.c index 5c9b8ae..f238914 100644 --- a/gcc-python-closure.c +++ b/gcc-python-closure.c @@ -39,6 +39,7 @@ gcc_python_closure_new_generic(PyObject *callback, PyObject *extraargs, PyObject } closure->callback = callback; + Py_INCREF(callback); // FIXME: we may want to pass in the event enum as well as the user-supplied extraargs

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

gcc-python-plugin November 2011