Documentation/Maemo Eclipse Tutorial/Eclipse Memory Profiling

(wikify slightly, use ambox)
m (Uninitialised values: add missing '>')
 
(2 intermediate revisions not shown)
Line 1: Line 1:
-
Even if a developer provides an extensible round of tests, a certain type of problems cannot be easily detected by unit tests. In addition, such kind of problems may break the application and also disturb the normal execution of the platform. For example, a memory leak is a potential problem on applications for mobile devices, because it certainly consumes an important and limited resource. Maemo SDK provides Valgrind, a profiling tool often used to detect memory-related errors, such as memory leaks and problems on memory allocation.
+
Even if a developer provides an extensible round of tests, a certain type of problems cannot be easily detected by unit tests. In addition, such kind of problems may break the application and also disturb the normal execution of the platform. For example, a memory leak is a potential problem on applications for mobile devices, because it certainly consumes an important and limited resource. Maemo SDK provides [[Documentation/devtools/maemo5/valgrind|Valgrind]], a profiling tool often used to detect memory-related errors, such as memory leaks and problems on memory allocation.
Maemo IDE integration ESbox product provides support for profiling applications by using Valgrind memory profiler. This tutorial starts with a quick introduction to the Valgrind profiling tool. After that, it demonstrates how you can profile C/C++, Qt4 and Python applications at ESbox using the Valgrind memory profiler tool integration. Finally, you can see how to interpret Valgrind output at ESbox.
Maemo IDE integration ESbox product provides support for profiling applications by using Valgrind memory profiler. This tutorial starts with a quick introduction to the Valgrind profiling tool. After that, it demonstrates how you can profile C/C++, Qt4 and Python applications at ESbox using the Valgrind memory profiler tool integration. Finally, you can see how to interpret Valgrind output at ESbox.
Line 24: Line 24:
: It detects memory-management bugs on application. Memcheck keeps track of the validity (if a memory location is properly initialized) and addressability (if the memory address points to a valid memory block).
: It detects memory-management bugs on application. Memcheck keeps track of the validity (if a memory location is properly initialized) and addressability (if the memory address points to a valid memory block).
-
Since Valgrind tools instrument the code in order to obtain more accurate information about application execution, Memcheck is often used because it provides detailed information about memory problems, which are very common in mobile application development. It can detect many memory-related errors that are common in C and C++ programs and that can lead to crashes and unpredictable behaviour. Memcheck is the default tool on Maemo environment. The <span><font face="monospace">-leak-check</font></span> option turns on the detailed memory leak detector. Memcheck will issue messages about memory errors and leaks that it detects.
+
Since Valgrind tools instrument the code in order to obtain more accurate information about application execution, Memcheck is often used because it provides detailed information about memory problems, which are very common in mobile application development. It can detect many memory-related errors that are common in C and C++ programs and that can lead to crashes and unpredictable behaviour. Memcheck is the default tool on Maemo environment. The <code>-leak-check</code> option turns on the detailed memory leak detector. Memcheck will issue messages about memory errors and leaks that it detects.
== Valgrind and IDE Integration ==
== Valgrind and IDE Integration ==
-
ESbox provides support only for the Valgrind memory (Memcheck) profiler tool for C/C++, Qt4 and Python projects. It uses the command <span><font face="monospace">valgrind -q -tool=memcheck -leak-check=yes</font></span> to run Valgrind memory check tool (Memcheck) with detailed memory leak detector. This section shows how to profile your applications at ESbox and how to analyse profiling results.
+
ESbox provides support only for the Valgrind memory (Memcheck) profiler tool for C/C++, Qt4 and Python projects. It uses the command <code>valgrind -q -tool=memcheck -leak-check=yes</code> to run Valgrind memory check tool (Memcheck) with detailed memory leak detector. This section shows how to profile your applications at ESbox and how to analyse profiling results.
=== Installation ===
=== Installation ===
-
You do not need to care about Valgrind installation. Once ESbox starts to profile your application with Valgrind, the system checks if necessary packages are properly installed in the environment. If they are not, ESbox installs and configure them. All these steps are performed by Validation Packages wizards, as shown in figure [[#fig:eclipse_memory_profiling_install-valgrind|10.1]].
+
You do not need to care about Valgrind installation. Once ESbox starts to profile your application with Valgrind, the system checks if necessary packages are properly installed in the environment. If they are not, ESbox installs and configure them. All these steps are performed by Validation Packages wizards, as shown in figure [[#figure-eclipse_memory_profiling_install-valgrind|10.1]].
-
{| summary="Installing Valgrind at target"
+
<div id="figure-eclipse_memory_profiling_install-valgrind">
-
|+ align="BOTTOM" |'''Figure 10.1:''' Installing Valgrind at target
+
[[Image:install-valgrind.png|frame|center|alt=Screenshot of Valgrind installation|Figure 10.1: Installing Valgrind to environment]]
-
|-
+
</div>
-
|
+
-
[[Image:install-valgrind.png|Image install-valgrind]]
+
-
|}
+
== Profiling your Maemo application with Valgrind ==
== Profiling your Maemo application with Valgrind ==
Line 47: Line 44:
}}
}}
-
At first, you must create a C/C++, Qt4 or Python Maemo Project and select an X86 Scratchbox target. To run Valgrind, right-click on the application (binary for C/C++ and Qt4 projects or the Python script for Python projects) and select '''Profile As &gt; Local Valgrind Memory Profiler''', figure [[#fig:eclipse_memory_profiling_valgrind-menu|10.2]]. If the tool is not properly installed at the selected target, the system can automatically install it for you, figure [[#fig:eclipse_memory_profiling_install-valgrind|10.1]].
+
At first, you must create a C/C++, Qt4 or Python Maemo Project and select an X86 Scratchbox target. To run Valgrind, right-click on the application (binary for C/C++ and Qt4 projects or the Python script for Python projects) and select '''Profile As &gt; Local Valgrind Memory Profiler''', figure [[#figure-eclipse_memory_profiling_valgrind-menu|10.2]]. If the tool is not properly installed at the selected target, the system can automatically install it for you, figure [[#figure-eclipse_memory_profiling_install-valgrind|10.1]].
-
{| summary="Profiling application with Valgrind"
+
<div id="figure-eclipse_memory_profiling_install-valgrind">
-
|+ align="BOTTOM" |'''Figure 10.2:''' Profiling application with Valgrind
+
[[Image:valgrind-menu.png|frame|center|alt=Screenshot of Valgrind profiling an application|Figure 10.2: Profiling an application with Valgrind]]
-
|-
+
</div>
-
|
+
-
[[Image:valgrind-menu.png|Image valgrind-menu]]
+
-
|}
+
-
ESbox runs your application on the Maemo SDK X86 target and the result is shown in the Valgrind view, figure [[#fig:eclipse_memory_profiling_valgrind-view|10.3]]. All problems found by Valgrind are shown as a tree: problems as parent nodes and their details as children nodes. Valgrind error messages are explained at chapter [[#Analyzing results]].
+
ESbox runs your application on the Maemo SDK X86 target and the result is shown in the Valgrind view, figure [[#figure-eclipse_memory_profiling_valgrind-view|10.3]]. All problems found by Valgrind are shown as a tree: problems as parent nodes and their details as children nodes. Valgrind error messages are explained in the [[#Analyzing results]] chapter.
-
{| summary="Valgrind View"
+
<div id="figure-eclipse_memory_profiling_valgrind-view">
-
|+ align="BOTTOM" |'''Figure 10.3:''' Valgrind View
+
[[Image:valgrind-view.png|frame|center|alt=Screenshot of Valgrind view|Figure 10.3: Valgrind View]]
-
|-
+
</div>
-
|
+
-
[[Image:valgrind-view.png|Image valgrind-view]]
+
-
|}
+
{{ambox
{{ambox
Line 75: Line 66:
* Accessing memory locations that must not be accessed, for example, overrunning and underrunning heap blocks, overrunning the top of the stack, and accessing memory after it has been freed.
* Accessing memory locations that must not be accessed, for example, overrunning and underrunning heap blocks, overrunning the top of the stack, and accessing memory after it has been freed.
* Using undefined values, that is, values that have not been initialised, or that have been derived from other undefined values.
* Using undefined values, that is, values that have not been initialised, or that have been derived from other undefined values.
-
* Incorrect freeing of heap memory, such as double-freeing heap blocks, or mismatched use of <span><font face="monospace">malloc/new/new[]</font></span> versus <span><font face="monospace">free/delete/delete[]</font></span>.
+
* Incorrect freeing of heap memory, such as double-freeing heap blocks, or mismatched use of <code>malloc</code>/<code>new</code>/<code>new[]</code> versus <code>free</code>/<code>delete</code>/<code>delete[]</code>.
-
* Overlapping src and dst pointers in <span><font face="monospace">memcpy</font></span> and related functions.
+
* Overlapping src and dst pointers in <code>memcpy</code> and related functions.
* Memory leaks.
* Memory leaks.
Line 83: Line 74:
==== Heap block overrun ====
==== Heap block overrun ====
-
A heap block overrun happens when you try to access (read/write) a block on heap memory that was not properly allocated. For example, an array of 10 integers is created (e.g, <span><font face="monospace">array[10]</font></span>), but you try to access the 11th element, which does not exist. Most C compilers do not warn about this problem, and it is often a potential source of bugs. Most error messages look like the following one, which describes the heap block overrun problem, figure [[#fig:eclipse_memory_profiling_heap-block|10.4]].
+
A heap block overrun happens when you try to access (read/write) a block on heap memory that was not properly allocated. For example, an array of 10 integers is created (e.g, <code>array[10]</code>), but you try to access the 11th element, which does not exist. Most C compilers do not warn about this problem, and it is often a potential source of bugs. Most error messages look like the following one, which describes the heap block overrun problem, figure [[#figure-eclipse_memory_profiling_heap-block|10.4]].
-
{| summary="Heap block overrun"
+
<div id="figure-eclipse_memory_profiling_heap-block">
-
|+ align="BOTTOM" |'''Figure 10.4:''' Heap block overrun
+
[[Image:valgrind-1.png|frame|center|alt=Screenshot of Valgrind showing heap a block overrun|'''Figure 10.4:''' Heap block overrun]]
-
|-
+
</div>
-
|
+
-
[[Image:valgrind-1.png|Image valgrind-1]]
+
-
|}
+
You must read each error message carefully to understand its information.
You must read each error message carefully to understand its information.
-
* The number between <span><font face="monospace"><nowiki>==</nowiki></font></span> symbols is the process ID (for example, <span><font face="monospace">7068</font></span>).
+
* The number between <code>==</code> symbols is the process ID (for example, <code>7068</code>).
-
* The parent node (<span><font face="monospace">Invalid read of size 4</font></span>) tells you what kind of error it is. Here, this message says that the program performed an illegal 4-byte read of address <span><font face="monospace">0x406CDC7</font></span>, which, as far as Memcheck can tell, is not a valid stack address, nor corresponds to any current heap blocks or recently freed heap blocks.
+
* The parent node (<code>Invalid read of size 4</code>) tells you what kind of error it is. Here, this message says that the program performed an illegal 4-byte read of address <code>0x406CDC7</code>, which, as far as Memcheck can tell, is not a valid stack address, nor corresponds to any current heap blocks or recently freed heap blocks.
* Details of the problems are shown as children nodes: a stack trace tells you where the problem occurred.
* Details of the problems are shown as children nodes: a stack trace tells you where the problem occurred.
-
* The code addresses (for example, <span><font face="monospace">0x406CDC7</font></span>) are usually unimportant, but occasionally crucial for tracking down weird bugs.
+
* The code addresses (for example, <code>0x406CDC7</code>) are usually unimportant, but occasionally crucial for tracking down weird bugs.
* Some error messages have a second component which describes the memory address involved.
* Some error messages have a second component which describes the memory address involved.
==== Memory leaks ====
==== Memory leaks ====
-
Another kind of common error is the memory leak, which happens when you do not free previously allocated memory resources, for example arrays. For memory leaks, Valgrind outputs the following messages, figure [[#fig:eclipse_memory_profiling_memory-leak|10.5]].
+
Another kind of common error is the memory leak, which happens when you do not free previously allocated memory resources, for example arrays. For memory leaks, Valgrind outputs the following messages, figure [[#figure-eclipse_memory_profiling_memory-leak|10.5]].
-
{| summary="Memory leak"
+
<div id="figure-eclipse_memory_profiling_memory-leak">
-
|+ align="BOTTOM" |'''Figure 10.5:''' Memory leak
+
[[Image:valgrind-2-memory-leak.png|frame|center|alt=Screenshot of Eclipse profiling a memory leak|'''Figure 10.5:''' Memory leak]]
-
|-
+
</div>
-
|
+
-
[[Image:valgrind-2-memory-leak.png|Image valgrind-2-memory-leak]]
+
-
|}
+
Message details tells you where the memory leak is happening. However, it does not tell you why the memory leaks. Among few kinds of leaks, there are two important categories.
Message details tells you where the memory leak is happening. However, it does not tell you why the memory leaks. Among few kinds of leaks, there are two important categories.
-
* <span><font face="monospace">Definitely lost</font></span><nowiki>: when the program is really leaking memory. </nowiki>
+
* <code>Definitely lost</code>: when the program is really leaking memory.
-
* <span><font face="monospace">Probably lost</font></span><nowiki>: when the program is possibly leaking memory. </nowiki>
+
* <code>Probably lost</code>: when the program is possibly leaking memory.
==== Uninitialised values ====
==== Uninitialised values ====
-
Memory check also reports uses of uninitialised values, commonly with the message conditional jump or move depends on uninitialised value(s), figure [[#fig:eclipse_memory_profiling_unini-values|10.6]].
+
Memory check also reports uses of uninitialised values, commonly with the message conditional jump or move depends on uninitialised value(s), figure [[#figure-eclipse_memory_profiling_unini-values|10.6]].
-
{| summary="Uninitialised values"
+
<div id="figure-eclipse_memory_profiling_unini-values">
-
|+ align="BOTTOM" |'''Figure 10.6:''' Uninitialised values
+
[[Image:valgrind-3-uninitialised-values.png|frame|center|alt=Screenshot of Eclipse profiling uninitialised values|Figure 10.6: Uninitialised values]]
-
|-
+
</div>
-
|
+
-
[[Image:valgrind-3-uninitialised-values.png|Image valgrind-3-uninitialised-values]]
+
-
|}
+
An uninitialised-value error is reported when your program uses a value which has not been initialised. In our example, the undefined value was used somewhere inside the <code>_dl_relocate_object</code>.
An uninitialised-value error is reported when your program uses a value which has not been initialised. In our example, the undefined value was used somewhere inside the <code>_dl_relocate_object</code>.
Line 131: Line 113:
==== System calls with invalid parameters ====
==== System calls with invalid parameters ====
-
Another common error is a system call with invalid parameters, figure [[#fig:eclipse_memory_profiling_invalid-parameter|10.7]].
+
Another common error is a system call with invalid parameters, figure [[#figure-eclipse_memory_profiling_invalid-parameter|10.7]].
-
{| summary="Invalid parameters"
+
<div id="figure-eclipse_memory_profiling_invalid-parameter">
-
|+ align="BOTTOM" |'''Figure 10.7:''' Invalid parameters
+
[[Image:valgrind-4.png|frame|center|alt=Screenshot of Eclipse profiling invalid parameters|Figure 10.7: Invalid parameters]]
-
|-
+
</div>
-
|
+
-
[[Image:valgrind-4.png|Image valgrind-4]]
+
-
|}
+
-
This error message shows that there is some uninitialised memory pointed to by <span><font face="monospace">buf</font></span> System call parameter.
+
This error message shows that there is some uninitialised memory pointed to by <code>buf</code> System call parameter.
There are also other kinds of error messages not listed at this document. More information about them can be found at ''Valgrind Manual'' <ref name=r5>Valgrind Manual. http://valgrind.org/docs/manual/manual.html</ref>.
There are also other kinds of error messages not listed at this document. More information about them can be found at ''Valgrind Manual'' <ref name=r5>Valgrind Manual. http://valgrind.org/docs/manual/manual.html</ref>.

Latest revision as of 11:39, 11 February 2011

Even if a developer provides an extensible round of tests, a certain type of problems cannot be easily detected by unit tests. In addition, such kind of problems may break the application and also disturb the normal execution of the platform. For example, a memory leak is a potential problem on applications for mobile devices, because it certainly consumes an important and limited resource. Maemo SDK provides Valgrind, a profiling tool often used to detect memory-related errors, such as memory leaks and problems on memory allocation.

Maemo IDE integration ESbox product provides support for profiling applications by using Valgrind memory profiler. This tutorial starts with a quick introduction to the Valgrind profiling tool. After that, it demonstrates how you can profile C/C++, Qt4 and Python applications at ESbox using the Valgrind memory profiler tool integration. Finally, you can see how to interpret Valgrind output at ESbox.

Image:Ambox_notice.png
Valgrind memory profiling cannot be used with Maemo PluThon Eclipse IDE. Valgrind profiler be run only with Scratchbox and Maemo SDK for x86 binaries and Maemo PluThon does not support those (it is on-device development environment for Maemo devices).

Contents

[edit] Valgrind Memory Profiling

Valgrind is an Open Source suite of tools for debugging and profiling Linux programs. For Maemo environment it i susable only for x86 binaries. It is a powerful and easy-to-use tool that can save you a lot of debugging time. Since Valgrind works by simulating the x86 processor and instrumenting the binaries on the fly, it can control the program execution completely. On the downside, it makes program execution 10-300 times slower than normal, depending on the used Valgrind tools, and can take a huge amount of memory.

The Valgrind tool suite provides a number of debugging and profiling tools:

Cachegrind
It is a profiler for cache memory. Cachegrind performs simulation of the I1, D1 and L2 caches in order to detect cache misses, wrong references and much more.
Callgrind
It is an extension of Cachegrind. It provides the same information as Cachegrind plus extra information about callgraphs.
Massif
It provides profilling information about heap memory by taking regular snapshots of a program's heap.
Helgrind
It is a useful tool to detect race conditions on multithreading applications, such as deadlocks.
Memcheck
It detects memory-management bugs on application. Memcheck keeps track of the validity (if a memory location is properly initialized) and addressability (if the memory address points to a valid memory block).

Since Valgrind tools instrument the code in order to obtain more accurate information about application execution, Memcheck is often used because it provides detailed information about memory problems, which are very common in mobile application development. It can detect many memory-related errors that are common in C and C++ programs and that can lead to crashes and unpredictable behaviour. Memcheck is the default tool on Maemo environment. The -leak-check option turns on the detailed memory leak detector. Memcheck will issue messages about memory errors and leaks that it detects.

[edit] Valgrind and IDE Integration

ESbox provides support only for the Valgrind memory (Memcheck) profiler tool for C/C++, Qt4 and Python projects. It uses the command valgrind -q -tool=memcheck -leak-check=yes to run Valgrind memory check tool (Memcheck) with detailed memory leak detector. This section shows how to profile your applications at ESbox and how to analyse profiling results.

[edit] Installation

You do not need to care about Valgrind installation. Once ESbox starts to profile your application with Valgrind, the system checks if necessary packages are properly installed in the environment. If they are not, ESbox installs and configure them. All these steps are performed by Validation Packages wizards, as shown in figure 10.1.

File:Install-valgrind.png
Figure 10.1: Installing Valgrind to environment

[edit] Profiling your Maemo application with Valgrind

Image:Ambox_notice.png
Valgrind does not run on ARM Linux platform. Therefore, you can only use Valgrind with X86 Maemo SDK rootstraps.

At first, you must create a C/C++, Qt4 or Python Maemo Project and select an X86 Scratchbox target. To run Valgrind, right-click on the application (binary for C/C++ and Qt4 projects or the Python script for Python projects) and select Profile As > Local Valgrind Memory Profiler, figure 10.2. If the tool is not properly installed at the selected target, the system can automatically install it for you, figure 10.1.

File:Valgrind-menu.png
Figure 10.2: Profiling an application with Valgrind

ESbox runs your application on the Maemo SDK X86 target and the result is shown in the Valgrind view, figure 10.3. All problems found by Valgrind are shown as a tree: problems as parent nodes and their details as children nodes. Valgrind error messages are explained in the #Analyzing results chapter.

File:Valgrind-view.png
Figure 10.3: Valgrind View
Image:Ambox_notice.png
If you are planning to run the Valgrind profiler more than once, you will probably need to clear current Valgrind view results before running the current program. For that, you just need to click the Delete Current Sample button on the Valgrind view toolbar. If you do not, current results are merged with the older ones.

[edit] Analyzing results

During Valgrind memory profiling, different problems can be detected, such as:

  • Accessing memory locations that must not be accessed, for example, overrunning and underrunning heap blocks, overrunning the top of the stack, and accessing memory after it has been freed.
  • Using undefined values, that is, values that have not been initialised, or that have been derived from other undefined values.
  • Incorrect freeing of heap memory, such as double-freeing heap blocks, or mismatched use of malloc/new/new[] versus free/delete/delete[].
  • Overlapping src and dst pointers in memcpy and related functions.
  • Memory leaks.

This section presents a quick summary about some kinds of Valgrind error messages and how to interpret them.

[edit] Heap block overrun

A heap block overrun happens when you try to access (read/write) a block on heap memory that was not properly allocated. For example, an array of 10 integers is created (e.g, array[10]), but you try to access the 11th element, which does not exist. Most C compilers do not warn about this problem, and it is often a potential source of bugs. Most error messages look like the following one, which describes the heap block overrun problem, figure 10.4.

File:Valgrind-1.png
Figure 10.4: Heap block overrun

You must read each error message carefully to understand its information.

  • The number between == symbols is the process ID (for example, 7068).
  • The parent node (Invalid read of size 4) tells you what kind of error it is. Here, this message says that the program performed an illegal 4-byte read of address 0x406CDC7, which, as far as Memcheck can tell, is not a valid stack address, nor corresponds to any current heap blocks or recently freed heap blocks.
  • Details of the problems are shown as children nodes: a stack trace tells you where the problem occurred.
  • The code addresses (for example, 0x406CDC7) are usually unimportant, but occasionally crucial for tracking down weird bugs.
  • Some error messages have a second component which describes the memory address involved.

[edit] Memory leaks

Another kind of common error is the memory leak, which happens when you do not free previously allocated memory resources, for example arrays. For memory leaks, Valgrind outputs the following messages, figure 10.5.

File:Valgrind-2-memory-leak.png
Figure 10.5: Memory leak

Message details tells you where the memory leak is happening. However, it does not tell you why the memory leaks. Among few kinds of leaks, there are two important categories.

  • Definitely lost: when the program is really leaking memory.
  • Probably lost: when the program is possibly leaking memory.

[edit] Uninitialised values

Memory check also reports uses of uninitialised values, commonly with the message conditional jump or move depends on uninitialised value(s), figure 10.6.

File:Valgrind-3-uninitialised-values.png
Figure 10.6: Uninitialised values

An uninitialised-value error is reported when your program uses a value which has not been initialised. In our example, the undefined value was used somewhere inside the _dl_relocate_object.

[edit] System calls with invalid parameters

Another common error is a system call with invalid parameters, figure 10.7.

File:Valgrind-4.png
Figure 10.7: Invalid parameters

This error message shows that there is some uninitialised memory pointed to by buf System call parameter.

There are also other kinds of error messages not listed at this document. More information about them can be found at Valgrind Manual [1].

[edit] References

  1. Valgrind Manual. http://valgrind.org/docs/manual/manual.html