Menu
Is free
registration
home  /  Installation and configuration/ Imaginary profiles php. Profiling and debugging php applications with xhprof

Imaginary profiles php. Profiling and debugging php applications with xhprof

Profiling PHP code

Sooner or later, each of us is faced with legacy code and its optimization. Debugger and profiler in such a situation - best helpers programmer. For those who work with PHP, thanks to Derick Rethans, there is a good tool - xDebug. There is a lot of information regarding xDebug even in runet, so this article will not be about it.

Having stumbled upon the mention of a profiler for PHP, I immediately thought about xDebug (I have long forgotten about proprietary tools from Zend), but this time I was mistaken - we will talk about XHProf.
XHProf

This profiler was developed specifically for Facebook, and source it was opened in March 2009.

The installation went quite quickly and smoothly.
wget pecl.php.net/get/xhprof-0.9.2.tgz
tar xvf xhprof-0.9.2.tgz
cd xhprof-0.9.2 / extension /
phpize
./configure && make && make install
cd /usr/local/etc/php.d/
vim xhprof.ini
cd / usr / local /
vim header.php
vim footer.php
vim etc / php.ini
/etc/init.d/php-fpm restart
cp vhost.conf.template prof.my.conf
sed -i s / site / prof / prof.my.conf
vim prof.my.conf
/etc/init.d/nginx restart

Let's analyze the mentioned configs

Xhprof.ini
extension = / usr / local / lib / php / extensions / no-debug-non-zts-20090626 / xhprof.so
xhprof.output_dir = "/ home / max / www / profile /"

Prof.my.conf - nginks config - the most standard.

Server (
listen 80;
server_name prof.my;
charset utf8;

Root /usr/local/src/xhprof-0.9.2/xhprof_html;
location / (
index index.php;
}

Location ~ \ .php $ (
fastcgi_pass 127.0.0.1:12000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /usr/local/src/xhprof-0.9.2/xhprof_html/$fastcgi_script_name;
include fastcgi_params;

/Usr/local/src/xhprof-0.9.2/xhprof_html contains PHP sources that create a nice WEBGUI for the profiler.

So about the two main files:

Header.php


include_once "/usr/local/src/xhprof-0.9.2/xhprof_lib/utils/xhprof_lib.php";
include_once "/usr/local/src/xhprof-0.9.2/xhprof_lib/utils/xhprof_runs.php";
xhprof_enable (XHPROF_FLAGS_CPU + XHPROF_FLAGS_MEMORY);
}
}

Footer.php
if (isset ($ _ COOKIE ["xhprof"])) (
if (extension_loaded ("xhprof")) (
$ profiler_namespace = "myapp"; // namespace for your application
$ xhprof_data = xhprof_disable ();
$ xhprof_runs = new XHProfRuns_Default ();
$ run_id = $ xhprof_runs-> save_run ($ xhprof_data, $ profiler_namespace);

// url to the XHProf UI libraries (change the host name and path)
$ profiler_url = sprintf ("http://prof.my/index.php?run=%s&source=%s", $ run_id, $ profiler_namespace);
echo<<Profiler output
OUT;
}
}

Now we run any PHP script via the web and see in the upper left corner a link to the profiler output - this is what the prof.my host was created for.

Please note - I am using verification for COOKIE! With this verification, you can safely use the profiler on the production server - on real data and on real load.

The profiler web interface displays labels with information about each function and reports the following information:

  • Number of calls to each function
  • Wall-time, the time spent executing functions (including waiting for responses from sockets, filesystem, etc.).
  • CPU-time, time spent on executing functions (excluding waiting for responses from sockets, filesystem, etc.).
  • Memory usage
  • Peak memory usage

It is possible to sort the table by any of the parameters

Information on each function is divided into two more types Inclusive and Exclusive. Inclusive includes the digits used by child calls, while Exclusive does not. It is also possible, by clicking on the name of the function, to see information only about it and the functions from which it was called and which were called to it.

If GraphViz is installed on the system, the profiler will draw a call graph for you.

P.S. Without breaking traditions: this is my first post on habr.

UPD: repost in PHP.

With the help of profiling systems, you can collect information about which functions in php code consume more CPU time and RAM, that is, identify the slowest and most memory-intensive places in a php program.

xhprof

XHProf - PHP profiler developed by Facebook.

Installation:

Aptitude install php-pear pecl install xhprof-0.9.4 echo "extension = xhprof.so"> /etc/php5/mods-available/xhprof.ini ln -s /etc/php5/mods-available/xhprof.ini / etc /php5/conf.d/xhprof.ini apachectl restart

Files required for work are located in the directory / usr / share / php... However, not all, but only with php-code. It requires jquery and css to display the reports properly. They can be obtained from the repository on github:

Git clone https://github.com/facebook/xhprof.git

After that, add the following line to the php-script code in the place where data collection should start:

Xhprof_enable (XHPROF_FLAGS_CPU + XHPROF_FLAGS_MEMORY);

Parameters for data collection are indicated in brackets. In this case, data will be collected on the load on the processor and on the use of RAM. One more parameter is possible XHPROF_FLAGS_NO_BUILTINS when using which data on built-in functions is not collected.

$ xhprof_data = xhprof_disable (); include_once "xhprof_lib / utils / xhprof_lib.php"; include_once "xhprof_lib / utils / xhprof_runs.php"; $ xhprof_runs = new XHProfRuns_Default (); $ run_id = $ xhprof_runs-> save_run ($ xhprof_data, "xhprof_test"); echo "Report: http: //domain.tld/xhprof_html/index.php? run = $ run_id & source = xhprof_test"; echo "\ n";

In line $ run_id the quotes indicate the name of the profile, which can be set arbitrarily.

The processed result looks like this:

If you specify the parameter XHPROF_FLAGS_NO_BUILTINS, then you can see that the number of function calls is significantly reduced:

The table provides the following information:

Calls- the number of function calls,
Wall time- the total operating time of the function, including the waiting time for a response from external resources,
Cpu- how much time was spent on processing functions,
MemUse- how much RAM was used,
PeakMemUse- peak memory consumption.

The modifiers are:

Incl- inclusive - taking into account the calls of other functions from this function,
Excl- exclusive - excluding function calls.

In addition, information on the total processing time, memory used and the number of function calls is presented above the table.

Also XHProf allows you to build differential reports between two runs, which are indicated in red and green. With the help of such reports, you can get a clear picture of the improvements after each code change.

To get such a report, you need to use a link of the form:

http: //domain.tld/xhprof_html/index.php? run1 = run_id1 & run2 = run_id2 & source = xhprof_test

where run_id1 and run_id2- launch identifiers.

If you install Graphviz:

Aptitude install graphviz

Also for php profiler xhprof there are third-party web interfaces using databases:

xDebug

xDebug is a profiling PHP code debugger written by Derick Rethans.

Installation:

Yum install php5-xdebug

Then we edit the config:

Nano /etc/php5/mods-available/xdebug.ini

adding the lines to it:

Xdebug.profiler_enable = 1 xdebug.profiler_aggregate = On xdebug.profiler_output_dir = / tmp

Here we turn on the PHP profiler and specify the directory in which to put the profiles. Profiles are created with names like cachegrind.out. *

There is a webgrind web client: https://github.com/jokkedk/webgrind. It doesn't work too fast, but it allows you to quickly view small profiles. In fact, this is PHP code that needs to be cloned from github:

Git clone https://github.com/jokkedk/webgrind.git

a directory will be created webgrind, which you need to copy to the directory of any site and access it from the browser. Further, so that graphing in the configuration file works in Debian config.php you need to correct the path to the executable file graphviz... It should look like this:

Static $ dotExecutable = "/ usr / bin / dot";

In addition, you can correct the time zone:

Static $ defaultTimezone = "Europe / Moscow";

In the header, you can select a profile and check the box whether to take into account built-in functions. In the table itself, you can see the functions, the number of calls, the running time of the function itself and the time taking into account the waiting time. To go deeper into the functions, just click on the triangular arrow. In my case, with sufficiently large profiles (from several megabytes), the expectation of the result was unnecessarily high. Probably, for large enough profiles it is better to use local viewer programs.

The graph might look like this:

note that webgrind should not be used on production servers, since no authorization is provided, but at the same time there is access to the code of the files in php. Use at least basic Apache authorization if necessary.

There are also programs for analyzing profiles as under Linux:

About profiling

Profile data can help you improve your application, that is, achieve certain goals, for example, lower memory consumption, reduce page generation time, and so on.

The information in the profile is the starting point in optimization: it tells how long the result is generated, how much memory is used and how many function calls are made. With more detailed data, you can improve these metrics.

For example, if you are using a framework, then using some of the framework's functions can lead to calling several basic functions. If you are reading some data multiple times, then it might be worth saving the result to a variable.

Also, the profiler can help you understand where to use PHP caching, for example, using APCu or memcached.

First of all, it is worth optimizing the functions that take the most time to execute. After everything is optimized and it seems that there is nothing more to improve, it is worth sorting the functions by the number of calls and work on lowering it. Even if PHP is fast, it's worth considering whether you need to call functions that often?

When you encounter the following situations, it is worth considering caching:

  • Immutable functions are called inside the loop,
  • Some content is generated twice,
  • Content that doesn't change is generated every time,
  • Content is generated even if not used.

You shouldn't cache everything, as memory is also a valuable resource. Cache the data you are constantly accessing. Also, caching makes little sense if caching wastes more resources than it saves.

In addition to caching in code, do not forget about caching using the web server (), as well as on the client side. By using the correct headers, many requests can be resolved before they reach the server.

Over time, any PHP programmer is faced with the problem of poor performance of their application. This could be a slow loading of a specific page or a too long response from the API. And sometimes it is quite difficult to understand what is the reason for the brakes? Sometimes more complicated situations happen: on the production server, the api is very slow, but at the stand where the development is taking place, everything is fine. And go figure out what's going wrong. Debugging on a production server is an extreme despair, which, of course, is best left unchecked.

It is for such situations that special tools were invented, called application profilers. In the PHP world, this role is played by xDebug as well as xhprof. xhprof is a lighter, simpler, and more flexible tool and is therefore preferred. Interestingly, xhprof was developed by facebook back in 2009, however there is still no official php7 support from them and will not be anymore since facebook switched to HHVM. However, thanks to the large community of php developers, a fork has appeared that supports php7, the installation of which does not cause any difficulties.

Installation

First you need to actually install xhprof:

Git clone https://github.com/longxinH/xhprof xhprof cd xhprof / extension phpize ./configure --with-php-config = / usr / bin / php-config sudo make && sudo make install mkdir / var / tmp / xhprof

Extension = xhprof.so xhprof.output_dir = "/ var / tmp / xhprof"

The / var / tmp / xhprof folder must have write access, because profiling results will be saved there.

You can reload PHP-FPM and check if the extension is installed. It's trite, it can be done using the output of the phpinfo () function;

xhprof is installed, you can use it. The xhprof package includes a very user-friendly interface for analyzing profiling reports. xhprof allows you to build reports, both in text and graphical form. The xhprof installation folder contains the xhprof_html and xhprof_lib that we need. Xhprof_html folder - provides access to the GUI. xhprof_lib is a library for displaying and analyzing code. It is advisable to move the entire xhprof folder to / var / www / xhprof and configure a virtual host for it, for example xhprof.loc. Example for nginx:

Server (listen 80; server_name xhprof.loc; charset utf-8; root / var / www / xhprof / xhprof_html; index index.php; location / (try_files $ uri $ uri / /index.php?q=$uri&$args ;) location ~ \ .php (fastcgi_pass 127.0.0.1:9000; fastcgi_split_path_info ^ (. + \. php) (/.+) $; fastcgi_param SCRIPT_FILENAME $ document_root $ fastcgi_script_name; include fastcgi_params;))

You also need to remember to update the hosts file. Now, when we enter the URL xhprof.loc into the browser, we will be taken to the profiler's web interface, where the files generated by it will be accessed.

Now you can start directly profiling your code.

To enable the profiler, use the xhprof_enable () function, which takes the following flags as input:

  • XHPROF_FLAGS_CPU - for fixing processor statistics;
  • XHPROF_FLAGS_MEMORY - for memory;
  • XHPROF_FLAGS_NO_BUILTINS - to ignore built-in functions.

To disable the profiler, use the xhprof_disable () function. For convenience, let's write two scripts header.php and footer.php that perform these functions. header.php is included at the beginning of the profiled script, and footer.php is included at the end. footer.php also deals with saving the profiling data.

header.php: if (extension_loaded ("xhprof")) (include_once "/var/www/xhprof/xhprof_lib/utils/xhprof_lib.php"; include_once "/var/www/xhprof/xhprof_lib/utils/xhpprof_runs.php xhprof_enable (XHPROF_FLAGS_CPU);) footer.php: if (extension_loaded ("xhprof")) ($ profilerNamespace = "HERE_PROFILED_SCRIPT_NAME"; $ xhprofData = xhprof_shprofRunRun = xhprof_disable (); ($ xhprofData, $ profilerNamespace);)
Usage

Having connected header.php and footer.php to the profiled script, you can start: when the profiled script is executed, a file will be generated that will be saved in the / var / tmp / xhprof directory containing information about the script's operation. When you open the xhprof.loc web interface, this generated file will be available:


When you open the profiling file, detailed information about the operation of the application appears, the entire call stack:


What the columns mean:

  • Calls- number and percentage of function calls;
  • Incl. Wall time- execution time of a function with nested functions;
  • Excl. Wall time- execution time of a function without nested functions;
  • Incl. Cpu- processor time with nested functions;
  • Excl. Cpu- CPU time without nested functions;
  • Incl. MemUse- memory consumption with nested functions;
  • Excl. MemUse- memory consumption without nested functions;
  • Incl. PeakMemUse- maximum memory consumption with nested functions;
  • Excl. PeakMemUse- maximum memory consumption without nested functions.

If you follow the link, you will see a beautiful call tree with a visual indication of the most inhibited code. If this did not happen, then most likely you need to install the graphviz library:

Apt-get install graphviz

An example of a plotted graph:

In my case, the bottleneck is the interaction with the database.


Using xhprof on a production server

Initially, xhprof was developed specifically for the purpose of profiling code in battle, on production servers. There is simply no other free and effective tool for profiling php7 code in battle, so xhprof has no competitors. Specifically, I have experience using xhprof on a production server that processes a million requests per day. It uses php7 and no problems have been found yet. However, xhprof does not run for every request - too many profiling files would be generated. My profiler starts only if the request contains the header "XHPROF_ENABLE" and it is set to true. You can also use another strategy, for example, run the profiler randomly with a probability of, say, 1/1000. Then the picture will be clear enough too.


Output

Even though xhprof is not officially supported for php7, it still remains an indispensable tool for php developers.

An extension to PHP called Xdebug is available to assist in profiling PHP applications, as well as runtime debugging. When running the profiler, the output is written to a file in a binary format called "cachegrind". Applications are available on each platform to analyze these files. No application code changes are necessary to perform this profiling.

To enable profiling, install the extension and adjust php.ini settings. Some Linux distributions come with standard packages (e.g. Ubuntu "s php-xdebug package). In our example we will run the profile optionally based on a request parameter. This allows us to keep settings static and turn on the profiler only as needed.

# php.ini settings # Set to 1 to turn it on for every request xdebug.profiler_enable = 0 # Let "s use a GET / POST parameter to turn on the profiler xdebug.profiler_enable_trigger = 1 # The GET / POST value we will pass ; empty for any value xdebug.profiler_enable_trigger_value = "" # Output cachegrind files to / tmp so our system cleans them up later xdebug.profiler_output_dir = "/ tmp" xdebug.profiler_output_name = "cachegrind.out.% p"

Next use a web client to make a request to your application "s URL you wish to profile, e.g.

Http://example.com/article/1?XDEBUG_PROFILE=1

As the page processes it will write to a file with a name similar to

/tmp/cachegrind.out.12345

By default the number in the filename is the process id which wrote it. This is configurable with the xdebug.profiler_output_name setting.

Note that it will write one file for each PHP request / process that is executed. So, for example, if you wish to analyze a form post, one profile will be written for the GET request to display the HTML form. The XDEBUG_PROFILE parameter will need to be passed into the subsequent POST request to analyze the second request which processes the form. Therefore when profiling it is sometimes easier to run curl to POST a form directly.

Analyzing the Output

Once written the profile cache can be read by an application such as or Webgrind. PHPStorm, a popular PHP IDE, can also display this profiling data.

KCachegrind, for example, will display information including:

  • Functions executed
  • Call time, both itself and inclusive of subsequent function calls
  • Number of times each function is called
  • Call graphs
  • Links to source code

What to Look For

Obviously performance tuning is very specific to each application "s use cases. In general it" s good to look for:

  • Repeated calls to the same function you wouldn "t expect to see. For functions that process and query data these could be prime opportunities for your application to cache.
  • Slow-running functions. Where is the application spending most of its time? the best payoff in performance tuning is focusing on those parts of the application which consume the most time.

Note: Xdebug, and in particular its profiling features, are very resource intensive and slow down PHP execution. It is recommended to not run these in a production server environment.

FirePHP is an extension for firebug, which, in conjunction with its small php class, allows you to broadcast data from php, for example var_dump and other debugging information, to the firebug console. headers and does not litter the pages and does not break the logic of the application in any way.Official site: http://firephp.org/.

Main idea.

The general profiling algorithm is as follows:
  1. At the beginning of the page, enable profiling using xhprof_enable ()
  2. At the end of the page, turn off profiling using xhprof_disable () and save the collected data using save_run ()
  3. Next, using the firephp php class, we transfer the link to the profiling data to the client side
  4. In the firebug console "and open the information we need
  5. We are happy :)
I would also like to say that, of course, manually adding these functions to your php scripts is great. But I want this information to be always at hand during development, and at the same time does not get to the combat servers. We solve this problem as follows:

In our projects, in almost all scripts, a working file with a class loader, connecting functions and other necessary things is connected at the beginning. Therefore, we moved the inclusion of profiling into this file. And in order to be able to turn on / off the debug mode at will, we added a check for a configuration constant, plus wrapped these checks in some meta tags that are automatically removed when building the project. The same applies to disabling profiling and writing information to headers using firephp - these tasks are solved by one function that is called at the end of each php script and is also wrapped in meta tags. It looks like this:

// These constants are written in the application config file

/ ** Environment operation mode * * /
define ("APPLICATION_ENV", "dev"); // dev - debug | pro - production
/ ** Path to profiler * /
define ("XHPROF_ROOT", __DIR__. "/ExtProcs/debug/xhprof-0.9.2");

/***************************************************************************************
* Next, in the file that is loaded at the beginning of each script, start profiling
* DEV_START and DEV_END are our meta tags, everything between them is cut out during assembly
***************************************************************************************/

// - DEV_START
// - in debug mode, connect the debug libraries

// Load firephp
require_once (__ DIR__. "/includes/ExtProcs/debug/firephp/FirePHP.class.php");
// - load the profiler
"/xhprof_lib/utils/xhprof_lib.php");
require_once (XHPROF_ROOT. "/xhprof_lib/utils/xhprof_runs.php");
// Initialize profiling with the required flags. Detailed description of flags
// can be found at php.net/manual/ru/xhprof.constants.php
xhprof_enable (XHPROF_FLAGS_CPU + XHPROF_FLAGS_MEMORY);
}
// - DEV_END

// Well, this function is called at the end of each script
// Its call is also wrapped in DEV_START and DEV_END

/**
* Create a link to the profiling result and output it to the console
*/
function dev_boot_Down () (
if (APPLICATION_ENV === "dev") (
// Initialize the firephp instance
$ firephp = FirePHP :: getInstance (true);
// Turn off profiling and save the data
$ xhprof_data = xhprof_disable ();
$ xhprof_runs = new XHProfRuns_Default ();
$ run_id = $ xhprof_runs-> save_run ($ xhprof_data, "xhprof_testing");
// Form a link to the profiling data and write it to the console
$ link = "http: //". $ _SERVER ["HTTP_HOST"]. "/includes/ExtProcs/debug/xhprof-0.9.2/xhprof_html/index.php?run=($run_id)&source=xhprof_testing\n";
$ firephp-> info ($ link, "profiling data");
}
}


* This source code was highlighted with Source Code Highlighter.

I will not go into the details of installing these extensions, because everything is simple here. I will only say about some of the setup points. Xhproof provides only one configuration variable, xhprof.output_dir, which points to the folder where the profiling data will be saved. Therefore, make sure that the user from under which php-scripts are executed has write permissions to the specified directory. So put something like this in your php.ini:


extension = xhprof.so
xhprof.output_dir = "/ var / tmp / xhprof"

It is also not bad to put something like dot or Graphviz for drawing call graphs. I have Graphviz under MacOS X.

After completing the above procedures, we were able to open and look at the profiling of any of our scripts right in the browser at any time.