Engineering
established 10 years ago

I'm getting a little closer to fielding Miles By Motorcycle on the HipHopVM (a.k.a. hhvm) stack. I, however, ran into a last minute snag having to do with some disconcerting warning messages in the error log:

HipHop Warning: Invalid argument: function: not a valid callback array

hhvm is much better than PHP at generating meaningful warnings about potential issues in your code. I was a little troubled by this message, however, since it did not provide me a call stack or even a function name. I always suspect my own code and understandings first, so I was fairly convinced it was something I was doing. It turned out to be an hhvm issue in that the custom session close handler was getting called twice. I originally tried using GDB to track through the code, but this was getting me nowhere I switched to configuring the hhvm debugger.

Unfortunately, this was not straightforward since there are a few gotchas.

The hhvm debugger can be invoked in two ways. You can run it interactively to debug local scripts, which in my case is of little use, or you can debug requests coming in from the browser. The latter case, once it works, is extremely cool.

Misconceptions

From the documentation I had a few misconceptions:

  • There is a "-m debug" switch for hhvm to enable the debugger, but this does not, by itself allow you to debug server requests.

  • You configure the server to enable remote debugging but you must use a second instance of hhvm called with "-m debug" and connect it to the first.

  • You cannot, to the best of my knowledge, get access to the threads serving requests on the live site. You must set up a debugging "sandboxes".

  • "sandboxes" consist of a user name and a sandbox name and these must be included in the hostname part of the URL you use to pull the page. The sandbox name refers to an entry in a .hphp file in the users home directory.

  • The documentation shows the format of the .hphp file as [sandbox].path=... but the format is actually sandbox.path= where sandbox gets replaced by the name of the sandbox.

  • Each sandbox gets it's own document root, access log and error log that are completely separate from the main site. In my case I just created a symlink to my sites DOCUMENT_ROOT and called it done.

  • It does not appear that you have access to rewriting rules from sandboxes making debugging a full site somewhat challenging.

  • The user account that you run the debugger as (using the -u option) should match the name of your sandbox account to avoid a nasty hanging issue when running the server and debugger on the same machine.

Configuring the HHVM Server for Debugging

There are a few steps involved to configure the server.

Sandbox Account and Hostname

The sandbox user account and sandbox name must be included in the hostname part of web requests made to your sandbox server. i.e. http://sandbox_user-sandbox_name.something.com/test.php.

There are probably other ways to do it as the hhvm Debugger unit tests imply, but in my case it was easier to just add an entry in /etc/hosts for my sandbox account name.

Sandbox names are split on a - and have the form sandbox_user-sandbox_name. If no sandbox_name is included, then the name 'default' is used for the sandbox.

Since I'm only working on one site right now, I added a user 'sandbox' to my machine and added a sandbox.mbymc.com entry to /etc/hosts. (replace mbymc with whatever domain your machine is on)

127.0.0.1 sandbox.mbymc.com

If I were working on more than one codebase, I could set up separate sandboxes like 'sandbox-site1', 'sandbox-site2', etc.

The HHVM Server Config File

Refer to the runtime options Sandbox Environment section: https://github.com/facebook/hhvm/wiki/runtime-options#sandbox-environment

You have to add a few blocks to your config.hdf.

Sandbox

Add this block to your servers config.hdf:

Sandbox{SandboxMode = truePattern = ([^-]*(-[^-]*)?).mbymc.comHome = /homeConfFile = .hphp}

Based on my testing, it looks like SandBoxMode must be set to true for this to be enabled.

The idea behind the Pattern setting is that it's a regex that must separate out two matches, one for the user account part and one for the optional sandbox name part. As I mentioned above, if there is no sandbox name part, the name 'default' is used.

The Home path is prepended to whatever the account name is and is used to search for the .hphp file. For example, if the request from the browser contains sandbox-site1.mbymc.com, the .hphp file for that request will be looked for in /home/sandbox/.hphp.

Eval.Debugger

Add the following section after the Sandbox section:

Eval.Debugger
{
EnableDebugger = true
EnableDebuggerServer = true
Port = 8089
DefaultSandboxPath = /path/to/docroot
}

The Sandbox Configuration File .hphp

The sandbox configuration file resides in the home directory of whatever account you're using for your sandbox. In my case I created a /home/sandbox account for this purpose. I created the file

/home/sandbox/.hphp

This file contains name/value pairs that define your various sandboxes for that account.

default.path = www
default.log = error_log
default.accesslog = access_log

The path is relative to the home directory so I simply created a symlink from www to the DOCUMENT_ROOT of my site. You can have entries for multiple sandboxes. If you had an additional site, you could, for instance, add:

site1.path = www2
site1.log = site1_error.log
site1.accesslog = access_log

Testing the Sandbox

Fire up your newly configured hhvm server

hhvm --config /usr/local/etc/hhvm_debugger.hdf -m server

Create a test.php file in the DOCUMENT_ROOT of your sandbox. Have it do something like print hello:

<?php
print( "<p>Hello</p>" );

Now load the file from your browser. In my case, the url was http://sandbox.mbymc.com/test.php.

You should see some output on the page.

Using the hhvm Debugger

Make sure that you pull a page from your sandbox server BEFORE proceeding from this point. Once you have pulled a page, you can now attach a debugger to the sandbox.

Fire up the debugger in a separate terminal window.

Enter the following command:

hhvm -m debug -h localhost -u sandbox_user

where sandbox_user is whatever user you chose to host your sandbox.

NOTE: Specifying the right user seems to be key in making this work correctly. Originally I would run the debugger under whatever user account I was in at the time and I would notice that the debugger would hang indefinitely on the attach step below more times than not. After trial and error, it seems that specifying the sandbox user on the debugger command line (instead of listing and attaching to sandbox instances) avoids this problem.

You should see:

Welcome to HipHop Debugger!

Type "help" or "?" for a complete list of commands.

Connecting to localhost:8089...
Attaching to sandbox's default sandbox and pre-loading, please wait...
localhost>

NOTE: You may need to run with root permissions to avoid a permissions error here. YMMV.

Refer to the Debugger overview documents here:https://github.com/facebook/hhvm/blob/master/hphp/doc/debugger.start https://github.com/facebook/hhvm/blob/master/hphp/doc/debugger.refs

You can now set a breakpoint to fire at the beginning of every request.

break start

You should see:

Breakpoint 1 set start of request.

Enter:

continue

Now reload your sandbox page. You should also notice that the webpage is just hanging waiting for a response.

In the debugger you should see:

Web request /test.php started.

To let the request complete, you can enter the command:

continue

You will notice that the prompt does not return after you do this. Reload the sandbox page in your browser in order to trigger the breakpoint and get the prompt back.

You can also press CTRL-C to get the debugger prompt back. This works as long as the debugger is running as the same user as the sandbox (at least in the case where both are running on the same physical machine).

From this point forward the debugger works as you would expect. You can set breakpoints, inspect variables and even execute arbitrary code.


Conclusion

In crawling around using GDB when I was looking at the HHVM source one of the traces seemed to indicate HHVM was trying to call a NULL custom session close handler. So I created a test file which is included on this issue:https://github.com/facebook/hhvm/issues/1316. It's just a dummy custom session handler.

In the debugger, I made sure to set a start of request breakpoint.

break start

I loaded the page implementing my test_session_handler.php test and the page load was interrupted at the start.

Web request /test_session_handler.php started.

I could now set a breakpoint at the start of _session_close():

break test_session::_session_close()

Which replied:

Breakpoint 3 set upon entering test_session::_session_close()
But wont break until class test_session has been loaded.

Now at the prompt I could enter

continue

which caused the breakpoint to fire on session close which displayed:

13 {
14* print( "session close<br>" );
15 return true;

I entered continue a second time and instead of having the script complete the same breakpoint fired again clearly indicating hhvm was calling the custom session close handler twice on exit.

Resources

The debugger has surprisingly decent online help. Just enter "help" at the debugger prompt for an overview. You can also get help on individual commands such as "help break".

An overview article on using the debugger: http://labs.qandidate.com/blog/2013/10/29/debugging-php-applications-with-hhvm/

HHVM Runtime Configuration Options: https://github.com/facebook/hhvm/wiki/runtime-options#sandbox-environment

More docs on using the debugger:https://github.com/facebook/hhvm/blob/master/hphp/doc/debugger.start https://github.com/facebook/hhvm/blob/master/hphp/doc/debugger.refs

You can follow me on twitter: http://twitter.com/yermolamers

    You must be a member of this group to post comments.

    Please see the top of the page to join.

    Link Details