Usage example:

$sfManagedCache = new sfManagedFunctionCache(
  new sfFileCache(array(
    'cache_dir' => sfConfig::get('sf_external_cache_dir').'/sfManagedCache',
    'lifetime' => 86400))
);

$xml = $sfManagedCache->call(
  array('Webserice', 'call''),
  array($parameter1, $parameter2),
  1800
);

The tasks are used like this and should be set up as cron jobs:

./symfony managed_cache:refresh --application=webservice --env=prod
./symfony managed_cache:refresh --env=prod

The first call is for a specific application while the second one is for application-agnostic cache items (not for all items!).

This is not a fire-and-forget plugin. It requires planning and maintenance. Since the cached items are designed to never expire, carelessness can easily lead to problems. I will cleanup the plugin some more, write documentation and then release it.

1) The data was raw and needed to be reprocessed before we could use it. That meant parsing the XML, running it through the processor and then writing it out again. Too much effort to do on every request, so again data would have to be cached.

2) More importantly, the webservice performance was atrocious. Request times for a 4 item rss feed hovered around 5 seconds. A completely rendered page could require several calls to this service. Whether these long request times were due to problems on our end, their end or somewhere in between was unimportant for the moment. The results would have to be cached to be useful.

Naturally, I could have left the caching to the application that used the webservice, ie. parse the information provided and store the results in memcache. The problem with that approach is that memcache is a very simple cache, and paired with the long request times for the webservice you're bound to run into problems. No matter how scalable your application and infrastructure is, if a reasonably common request is taking 5, 10, 15 seconds to complete, you're going to see a thread pileup sooner or later, which is what will happen when the items in memcache expire. And don't even get me started on the degraded user experience.

The proper solution for this is a managed cache.  One important concept is prefetch, ie. refresh the cached data before it expires, so it doesn't matter how long it takes to create it. WP-Supercache has this feature and Varnish recently introduced this feature as well along with grace. Grace is the other concept, which allows you to continue serving up stale or expired data if the data source becomes unavailable. In most cases, serving up old data is better than no data at all.

Symfony doesn't have any plugins or classes or other facilities to deal with these kinds of things as of right now, so I decided to just deploy Varnish for the purpose of caching the modified webservice data. No development time and very little effort to deploy and working perfectly. Unfortunately, I have to abandon that setup because of "complexity concerns". So in the next couple of weeks, I'll probably write a managed cache plugin for symfony.

I needed to run symfony in a virtual subfolder under an existing site, so that the asset URLs generate by helpers still originate from the root folder while URLs generated from routes contain the virtual subfolder. For example:

I have a site xyz.com that is run on some legacy platform that uses the same sort of layout for assets as symfony (stylesheets in /css, etc.). All the traffic for xyz.com goes through a load-balancer that splits off requests which URL starts with the prefix /symfonyApp and sends them to the server(s) running the symfony app. Symfony needs to generate asset URLs that look like /images/logo.gif while generating action URLs looking like /symfonyApp/signup/index.

Analysis:

The reason I was able to make this work at all is that the helper functions and the routing classes use different ways to insert the prefix folder name. Helpers get something called "RelativeUrlRoot" from the request object. This value can be set in factories.yml. Otherwise, it's calculated on demand by taking whatever is in front of the script file name in the SCRIPT_NAME environment variable.

Routes, on the other hand, store the prefix internally, though it's initialized to "RelativeUrlRoot + ScriptName", unless you override it in factories.yml.

Solution:

One solution would be to set the desired prefix values in factories.yml. But that requires duplicating the desired environment, which would be a pain to replicate. Instead, I opted to modify the values in the script file prior to dispatching the request. This way seems to work smoothly.

<?php

$tmp = dirname(__FILE__);
ini_set('error_log', $tmp . '/../../log/frontend_virtual_prod-error.log');

require_once($tmp.'/../../config/ProjectConfiguration.class.php');

$configuration = ProjectConfiguration::getApplicationConfiguration('frontend', 'prod', false);
$context = sfContext::createInstance($configuration);

// set separate prefixes for assets and links
$context->getRequest()->setRelativeUrlRoot(");
$routing = $context->getRouting();
$routingOptions = $routing->getOptions();
$routingOptions['context']['prefix'] = '/symfonyApp';
$routing->initialize($context->getEventDispatcher(), $routing->getCache(), $routingOptions);

$context->dispatch();

So, I can't really say much yet, but some things have certainly gotten easier and more elegant. As long as the performance stays the same, I probably won't complain. No, I'll probably find something regardless.

The reason the test worked fine while the real process didn't is a not entirely unimportant weakness in the test framework. The individual requests simulated by functional tests are not isolated from one another like regular requests would be. They all happen inside the same request context, so there is no teardown at the end and no cleanup. There are certain things in PHP that you can't undo unless you finish a request. One of those things is registering functions.

So what happened was that I included a helper (collection of functions) only in the template of the first request in the process. That was no problem in the test run because the functions stayed defined throughout, whereas in the real world, they wouldn't.

I'm trying to figure out what would be the best way to address this problem. The most thorough way, isolating every request into its own CLI session, would be impractical. Although it would address more than this very specific problem. The less general but easier to implement way would be to write a parser that checks whether every function used in the templates is defined or included in the template it's used in.

When using a custom 404 module/action, calling sfController::forward() or sfController::redirect() or any other part of symfony that results in an sfStopException being thrown, this exception will not be caught and thus result in a fatal error. The same is true for all other exceptions as well, but I'm highlighting sfStopExceptions because they are part of symfony's normal operation and it can be reasonably expected they be caught.

Analysis:

The forward to the 404 action happens inside the printStackTrace() method of the sfError404Exception that is throw and immediately caught in sfFrontWebController::dispatch(). There are no try-catch blocks around either the forward() call nor the printStackTrace call to catch exceptions that are throw as part of symfony's normal operation, or any exceptions for that matter.

Stack trace:

PHP Fatal error:  Uncaught exception 'sfStopException' in /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php:294
Stack trace:
#0 /var/www/promotions/current/apps/frontend/modules/error/actions/actions.class.php(19): sfAction->redirect('http://…', 404)
#1 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(462): errorActions->executeError404(Object(sfWebRequest))
#2 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(1109): sfActions->execute(Object(sfWebRequest))
#3 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(1104): sfExecutionFilter->executeAction(Object(errorActions))
#4 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(1093): sfExecutionFilter->handleAction(Object(sfFilterChain), Object(errorActions))
#5 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(1189): sfExecutionFilter->execute(Object(sfFilterChain))
#6 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(1051): sfFilterChain->execute()
#7 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(1189): sfCommonFilter->execute(Object(sfFilterChain))
#8 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(1152): sfFilterChain->execute()
#9 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(1189): sfRenderingFilter->execute(Object(sfFilterChain))
#10 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(734): sfFilterChain->execute()
#11 /var/www/promotions/current/lib/symfony/exception/sfError404Exception.class.php(49): sfController->forward('error', 'error404')
#12 /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php(2530): sfError404Exception->printStackTrace()
#13 /var/www/promotions/current/lib/symfony/util/sfContext.class.php(164): sfFrontWebController->dispatch()
#14 /var/www/promotions/current/web/index.php(9): sfContext->dispatch()
#15 {main}
thrown in /var/www/promotions/current/cache/frontend/prod/config/config_core_compile.yml.php on line 294

Possible solutions:

One way to solve this problem would be to move the forward out of the printStackTrace() method since that strikes me as a weird place to put it in the first place. The other solution would be to place another try-catch block around either the forward or the printStackTrace() call to ensure that normal symfony behavior can occur without causing a fatal error.

I found an annoying rather impeding bug in symfony the other day, and submitted the follwing ticket:

Symfony's clear cache task will fail when trying to clear config cache files created by another user (ie. web server) when there are config files that are newer than the cached versions.

./symfony cc

>> cache Clearing cache type "all" for "frontend" app and "prod" env

Failed to write cache file "/var/www/promotions/current/cache/frontend/prod/config/config_settings.yml.php" generated from configuration file "config/settings.yml".

Analysis:

While cache folders are created with full permissions (777, or umask 000) the cache files are created with default permissions (644, or umask 022).

When the clear cache task executes, it iterates through all cached applications and environments and loads the configuration for each before doing the actual cleaning.

Because it loads the application configuration with debug mode turned on, sfConfigCache will check if any of the config files are newer than the cached version and reprocess each such file it finds.

Since the task runs under a user different from the owner of the config cache files, writing the new config cache file fails.

Stack trace:

at /var/www/promotions/current/lib/symfony/config/sfConfigCache.class.php:344
sfConfigCache->writeCacheFile at /var/www/promotions/current/lib/symfony/config/sfConfigCache.class.php:107
sfConfigCache->callHandler at /var/www/promotions/current/lib/symfony/config/sfConfigCache.class.php:194
sfConfigCache->checkConfig at /var/www/promotions/current/lib/symfony/config/sfApplicationConfiguration.class.php:113
sfApplicationConfiguration->initConfiguration at /var/www/promotions/current/lib/symfony/config/sfApplicationConfiguration.class.php:49
sfApplicationConfiguration->__construct at /var/www/promotions/current/lib/symfony/config/sfProjectConfiguration.class.php:342
sfProjectConfiguration::getApplicationConfiguration at /var/www/promotions/current/lib/symfony/task/cache/sfCacheClearTask.class.php:104
sfCacheClearTask->execute at /var/www/promotions/current/lib/symfony/task/sfBaseTask.class.php:77
sfBaseTask->doRun at /var/www/promotions/current/lib/symfony/task/sfTask.class.php:75
sfTask->runFromCLI at /var/www/promotions/current/lib/symfony/command/sfSymfonyCommandApplication.class.php:72
sfSymfonyCommandApplication->run at /var/www/promotions/current/lib/symfony/command/cli.php:20
include at /var/www/promotions/current/symfony:15

Proposed solutions:

Create the config cache files with full write permissions. Since the cache folders already have full write permissions and the sticky bit doesn't get set, everyone can delete cache files and then create another file with the same name. Therefore, it shouldn't make a difference from a security standpoint.

Alternatively, unlink the existing file before writing the new one.

Here's how to reproduce the problem:

1. clear cache manually: rm -rf cache/*
2. make a request through your browser to any application
3. change the mod time on any config file that is required to build the application configuration for that application: touch apps/frontend/config/settings.yml