《Yii快速应用程序开发(高手)》

Chapter 7. Let It Work While You Sleep – Reports and Job Queues

This project is all about reporting and scheduling. We will create a job queue to schedule resource intensive work for off-peak hours and a report on our data to demonstrate some reporting/presentation packages that you can use with Yii.

Mission Briefing

The idea of writing a job is to manage a bit of code that will consume a noticeable amount of resources to execute. It is something you do not need to execute immediately, so you can load the work, ask it to run at a time where it will least impact your site, and then check on the results later.

One type of process that fits this description is correspondence, and another type is report generation.

We will build a system for managing and scheduling jobs, and use it to schedule an e-mail sending job and a graphical reporting job that produces a chart like the one shown in the following screenshot:

Why Is It Awesome?

As this system is your personal system, you can use it to send e-mails or reports to you or perform any task you want, at any time you want. If you work on commercial systems that serve a large number of users, you cannot perform these actions so flexibly. This chapter will demonstrate some techniques and systems that you can use when you work on highly-available web systems.

Your Hotshot Objectives

In this project, we will cover the following tasks:

  • Reorganizing Menu Items
  • Scaffolding the Job Objects
  • Adding Job Registration
  • Adding Job Scheduling
  • Adding Job Processing
  • Creating and Registering a Job
  • Creating a Graphical Report
  • Displaying Graphical Report Output

Mission Checklist

This project assumes that you have a web development environment prepared. If you do not have one, the tasks in Project 1, Develop a Comic Book Database, will guide you through setting one up. In order to work this project, you will need to set up the project files that have been provided with the book. Refer to the Preface of the book for instructions on downloading these files. The files for this project include a Yii project directory with a database schema. To prepare for the project, follow these steps, replacing the username lomeara with your own username.

  1. Copy the project files into your working directory.
cpr ~/Downloads/project_files/Chapter\ 7/project_files ~/projects/ch7
  1. Make the directories that Yii uses web writeable. For example, by using the following command we change ownership of the directories so that our user owns them, but the web group, www-data, can read, write, and execute the directories and contents, as well.
cd ~/projects/ch7/
sudo chown -R lomeara:www-data protected/runtime assets protected/models protected/controllers protected/view
  1. Create a link in the webroot directory to the copied directory.
cd /opt/lampp/htdocs
sudo ln -s ~/projects/ch7 cbdb
  1. Import the project into NetBeans (remember to set the project URL to http://localhost/cbdb) and configure for Yii development with PHPUnit.
  2. Create a database named cbdb and load the database schema (~/projects/ch7/protected/data/schema.sql) into it.
  3. If you are not using the XAMPP stack or if your access to MySQL is password protected, you should review and update the Yii configuration file (in NetBeans: ch7 | Source Files | protected | config | main.php).

Note that the admin login to the web application is admin/test.

Reorganizing Menu Items

At the start, we have several administrative menu options for our site. We will reorganize them and add the new options for this chapter.

Engage Thrusters

  1. Edit ch7 | Source Files | protected | views | layouts | main.php and create a new top-level item to collect administrative tasks. Set visible to true, so that the new items render.
'items'=>array(
    array('label'=>'Home', 'url'=>array('/site/index'), 'visible' => true),
    array(
        'label'=>'Comic Books',
        'url'=>array('/book/index'),
        'items' => array(
            array('label'=>'Publishers', 'url'=>array('/publisher/index')),
            array('label'=>'WishList', 'url'=>array('/wish/index')),
            array('label'=>'Library', 'url'=>array('/library/index')),
        ),
        'authItemName' => 'WishlistAccess',
    ),
    array(
        'label'=>'Admin', 
        'url' => '',
        'visible' => true,     
    ),

The menu will now display an Admin option.

  1. Move the existing administrative items srbac, audit trail, and users into the array for this new Admin item:
array(
    'label'=>'Admin',
    'url' => '',
    'items' => array(
        array('label'=>'Srbac', 'url'=>array('/srbac'), 'authItemName' => 'Authority'),
        array('label'=>'AuditTrail', 'url'=>array('/auditTrail/admin'), 'authItemName' => 'Authority'),
        array('label'=>'Users', 'url'=>array('/user/index')),
    ),
    'visible' => true
),

The resulting menu will look like the following screenshot:

  1. Change the value of visible to check if access is allowed for any of the administrative menu items. Checking access for the UserIndex operation or Authority role covers the menu items for now.
 'visible' => Yii::app()->user->checkAccess('UserIndex')|| Yii::app()->user->checkAccess('Authority'),
  1. Add menu items for the new features that we will be creating later in this project.
array('label'=>'Jobs', 'url'=>array('/scheduledJob/index')),
  array('label'=>'Reports', 'url'=>array('/report/index')),

You will not see these new menu items, because the menu is rendered by YiiSmartMenu. We have not created an operation and assigned authorization for these new items, yet.

  1. Create operations ScheduledJobIndex and ReportIndex. See Project 5, Service Please – Integrating Service Data, for details on creating operations.
  1. Create the task jobsAndReports.
  1. Assign new operations ScheduledJobIndex and ReportIndex to the new task jobsAndReports.
  1. Assign the task jobsAndReports to the role admin.
  1. Now that our admin user is authorized to access the new actions, the new items will appear in the Admin drop-down menu.

Objective Complete - Mini Debriefing

In this task, we reorganized the site menu to collect all administrative tasks in a single category. We added new items, which we will flesh out in future tasks in this project. Because we are using YiiSmartMenu to render menu items only if the user is authorized to access them, we used the checkAccess function to determine whether or not to display the Admin menu item, and we had to set up new actions in the role-based access control system before they would render in the menu.

Classified Intel

To be thorough, add the new actions to the visible condition for the menu. This will ensure that if we reorganize the authorization hierarchy in the future, the Admin menu will still render for any user authorized to use one of the items in the list.

'visible' => Yii::app()->user->checkAccess('UserIndex') || 
                    Yii::app()->user->checkAccess('Authority') || 
                        Yii::app()->user->checkAccess('ReportIndex') ||
                            Yii::app()->user->checkAccess('ScheduledJobIndex') ,

Scaffolding the Job Objects

We will use two objects to manage the jobs. One is the job, which holds information about jobs registered with our system. The other is the scheduled job, which is the job queue entry. It represents a request to run a job, and contains the job results and any reported output when the job has finished running. In this section, we will scaffold both objects.

Engage Thrusters

  1. We have provided a table definition for the jobs – ready for your use. To load it in NetBeans, open an SQL command window for the cbdb database.
  2. Right-click on the command window and select Select in Projects.
  3. Navigate to the jobs SQL file, ch7 | Source Files | protected | data | job.sql, in the project window.
  4. Copy the contents from jobs.sql and paste it into the command window.
  5. Hit Shift + F6 to run the command, and check the SQL output for success.
  6. Do the same for the scheduled jobs schema ch7 | Source Files | protected | data | scheduled_job.sql.
  7. Make sure that the web server can write to the models, views, and controllers directory in your project. In Unix, we use the chown command to give write permissions to the www-data group.
cd ~lomeara/projects/ch7/protected
chown lomeara:www-data models/
chown lomeara:www-data views/
chown lomeara:www-data controllers/
  1. Use Gii to generate a model and CRUD from the job table and the scheduled job table.

You can now click on Jobs in the Admin menu to access the Jobs Index list page, but you will not yet see any job actions such as Create.

  1. In srbac, create operations for the CRUD actions. You already created one for ScheduledJobIndex in the previous task. You can go to Managing auth items | Autocreate Auth Items to create all of the operations at once. Remember to uncheck the option to create tasks, because we have already created a container task, jobsAndReports.
  2. In Srbac | Assign to users | Tasks, add all of the job operations to the jobsAndReports task.
  1. Replace index.php with admin.php in both the jobs and scheduledJobs view directories.
  2. In both JobController and ScheduledJobController, delete the actionIndex and accessRules functions. Remove the accessControl entry in the filter function. Rename actionAdmin to actionIndex. Search and replace admin with index.
  3. Now we will create a way to access the job list from the job queue. Edit ch7 | Source Files | protected | views | scheduledJob | index.php. Remove List ScheduledJob from the array, and add an entry to List Registered Jobs and Register Job.
$this->menu=array(
  array('label'=>'Schedule Job', 'url'=>array('create')),
  array('label'=>'List Registered Jobs', 'url'=>array('job/index')),
  array('label'=>'Register Job', 'url'=>array('job/create')),
);

This will result in the scheduledJob index looking similar to the following screenshot:

Objective Complete - Mini Debriefing

In this task, we have created tables to hold job data and our job queue. We used Gii to produce an initial scaffolding, and srbac to create access control entries. Finally, we made some general changes to the scaffolding and created a link in the interface between the two objects.

Adding Job Registration

Before we can add jobs to the queue, we need to be able to register a job with the system. We will build on the job scaffolding in this task to the point where we can manage the jobs registered with our site.

Prepare for Lift Off

  1. Create an extensions directory in your project.
mkdir ~/projects/ch7/protected/extensions
  1. Use wget to download the Yii extension quickdlgs from http://www.yiiframework.com/extension/quickdlgs/ as follows:
cd ~/Downloads
wget http://www.yiiframework.com/extension/quickdlgs/files/quickdlgs.1.2.zip

We are going to use the quickdlgs extension to add modal dialogs to create, add, and edit entries in our grids.

  1. Unzip the package in your project's extensions directory as follows:
cd ~/projects/ch7/protected/extensions/
unzip ~/Downloads/quickdlgs.1.2.zip
  1. Add the following entry to the import array in ch7 | Source Files | protected | config | main.php.
 'ext.quickdlgs.*'

Engage Thrusters

  1. Edit ch7 | Source Files | protected | views | job | index.php. Correct the breadcrumb.
$this->breadcrumbs=array(
    'Registered Jobs',
);

Change the action menu to link to the Scheduled Job list instead of the Job List, and remove the Create Job entry. We will be adding a button to perform this function shortly.

$this->menu=array(
    array('label'=>'List Scheduled Jobs', 'url'=>array('scheduledJob/index')),
);

Remove the ID field from the job grid.

The updated page will look like the following screenshot:

  1. In the same file (ch7 | Source Files | protected | views | job | index.php), add a create button, using the new extension iframeButton function, right before the grid is generated.
<div class="right">
  <?php
    EQuickDlgs::iframeButton(
      array(
        'controllerRoute' => 'create',
        'dialogTitle' => 'Create item',
        'dialogWidth' => 800,
        'dialogHeight' => 275,
        'openButtonText' => 'Register New Job',
        'closeButtonText' => 'Close',
        'closeOnAction' => true, //important to invoke the close action in the actionCreate
        'refreshGridId' => 'job-grid', //the grid with this id will be refreshed after closing
      )
    );
  ?>
</div>
<?php $this->widget('zii.widgets.grid.CGridView', array(

The screen with the new Register New Job button will look like the following screenshot:

You can reload the page and try the button at this point. It will produce a modal dialog that contains the entire job creation page. Also, if you create a record, the grid will not update to indicate that a new record has been added.

  1. To make the name field fit nicely into the modal dialog, change the size of the field in the form ch7 | Source Files | protected | views | job | _form.php.
<?php echo $form->textField($model,'name',array('size'=>40,'maxlength'=>64)); ?>
  1. We will still access the Create Job screen from the Job Scheduling page, because it might be convenient to add a job in the midst of scheduling jobs. To reuse the same view for both the modal dialog and the full-page view, we will want to maintain the existing information and make it look nice in modal form.
  2. Start by changing the action menu to contain links back to the Registered Jobs list and the Scheduled Jobs list in ch7 | Source Files | protected | views | job | create.php.
$this->menu=array(
    array('label'=>'List Registered Jobs', 'url'=>array('job/index')),
    array('label'=>'List Scheduled Jobs', 'url'=>array('scheduledJob/index')),
);

Change the index label in the breadcrumbs to show that it is for Registered Jobs.

$this->breadcrumbs=array(
    'Registered Jobs'=>array('index'),
    'Create',
);

Change the header on the page from H1 – Create Job to H6 – Register Job.

<h6>Register Job</h6>
  1. To stop the site template from rendering in the create dialog, replace the render call in the actionCreate function in the Job Controller (ch7| Source Files| protected| controllers| JobController.php) to EQuickDlgs::render, which will detect the source of the call as an Ajax request and renderPartial the create view.
EQuickDlgs::render( 'create',array(
    'model'=>$model,
));

Also, add brackets around the if save condition, add a call to checkDialogJsScript, and change the redirect to the admin view.

if($model->save()) {
    EQuickDlgs::checkDialogJsScript();
    $this->redirect(array('admin'));
}

Now, when you click on create from the grid, the create dialog will look like this:

  1. We will make the update view modal, also, so you can make the same changes to actionUpdate in the Job Controller. The resulting function will look like the following code snippet:
public function actionUpdate($id)
{
    $model=$this->loadModel($id);
 
    // Uncomment the following line if AJAX validation is //needed
    // $this->performAjaxValidation($model);
 
    if(isset($_POST['Job']))
    {
        $model->attributes=$_POST['Job'];
        if($model->save()) {
            EQuickDlgs::checkDialogJsScript();
            $this->redirect(array('admin','id'=>$model->id));
        }
    }
 
    EQuickDlgs::render('update',array(
        'model'=>$model,
    ));
}
  1. The modal update will not appear until we change the buttons in the job index view. Replace the array that contains CButtonColumn with the following array that uses EJuiDlgsColumn in ch7 | Source Files | protected | views | job | index.php.
'columns'=>array(
    'name',
    array(
        'class'=>'EJuiDlgsColumn',
        'updateDialog'=>array(
            'dialogWidth' => 580,
            'dialogHeight' => 250,
        ),
        'viewDialog'=>array(
            'dialogWidth' => 580,
            'dialogHeight' => 250,
        ),
    ),
),
  1. We must change the Update view like we changed the Create view to make the modal appearance nicer and the page view (although we are not currently using it) clearer.

Change the action menu to display links back to the Registered Jobs list and the Scheduled Jobs list, as well as Create and View actions.

$this->menu=array(
    array('label'=>'Create Job', 'url'=>array('create')),
    array('label'=>'View Job', 'url'=>array('view', 'id'=>$model->id)),
    array('label'=>'List Registered Jobs', 'url'=>array('job/index')),
    array('label'=>'List Scheduled Jobs', 'url'=>array('scheduledJob/index')),
);

Change the index label in the breadcrumbs to show that it is for Registered Jobs.

$this->breadcrumbs=array(
    'Registered Jobs'=>array('index'),
    $model->name=>array('view','id'=>$model->id),
    'Update',
);

Change the header on the page from H1 to H6.

<h6>Update Job <?php echo $model->id; ?></h6>
  1. Finally, change the job controller view action to produce a modal result.
    EQuickDlgs::render( 'view',array(
        'model'=>$this->loadModel($id),
    ));
}

Objective Complete - Mini Debriefing

We have fleshed out the job controls and consolidated them into a single page using Ajax, CGridView, and the quickdlgs extension.

Adding Job Scheduling

In this task, we will clean up and customize the scaffolded job scheduling pages, to make it easier for us to add and test job scheduling and processing in the following tasks.

Prepare for Lift Off

  1. Download the Yii extension timepicker from http://www.yiiframework.com/extension/timepicker/.
cd ~/Downloads
wget http://www.yiiframework.com/extension/timepicker/files/timepicker5.zip
  1. Unzip the package in your project's extensions directory.
cd ~/projects/ch7/protected/extensions/
unzip ~/Downloads/timepicker5.zip
  1. Add the following entry to the import array in ch7 | Source Files | protected | config | main.php.
'ext.timepicker.*'

Engage Thrusters

  1. Start by adding a class variable named job_name to the ScheduledJob model in Source Files | protected | models | ScheduledJob.php. This change is similar to the work we did on the user form in Project 3, Access All Areas – Users and Logins.
class ScheduledJob extends CActiveRecord
{
    public $job_name;
  1. We will add a job drop-down to the scheduled job form, but before we do that, we have made enough drop-down functions to generalize our work. Create a class file in the components directory (Source Files | protected | components) named SelectableActiveRecord.php containing the following:
<?php
class SelectableActiveRecord extends CActiveRecord {
    public function getOptions()
    {
        return CHtml::listData($this->findAll(),'id','name');
    }
}
  1. Edit Source Files | protected | models | Job.php. Change the base class from CActiveRecord to SelectableActiveRecord.
class Job extends SelectableActiveRecord
  1. We will do a quick retro-fit of Grade and Type models to use this base class. Remove the function getGradeOptions from Source Files | protected | models | Grade.php and the function getTypeOptions from Source Files | protected | models | Type.php. Change base class of each from CActiveRecord to SelectableActiveRecord. Then, update Source Files | protected | views | book | _form.php and change the drop-down list of both Grade and Type to use getOptions.
  2. Now update the form for adding a scheduled job. Open Source Files | protected | views | scheduledJob | _form.php.

Move the job_id field to the top of the page and change the field type for the job_id field from textField to dropDownList using the new getOptions function.

<?php echo $form->labelEx($model,'job_id'); ?>
<?php echo $form->dropDownList($model, 'job_id', Job::model()->getOptions()); ?>
<?php echo $form->error($model,'job_id'); ?>

Change the field type for the active field from textField to checkbox.

<?php echo $form->labelEx($model,'active'); ?>
<?php echo $form->checkbox($model,'active'); ?>
<?php echo $form->error($model,'active'); ?>

Replace the scheduled_time field with the timepicker widget.

<div class="row">
    <?php echo $form->labelEx($model,'scheduled_time'); ?>
    <?php $this->widget('ext.timepicker.timepicker', array(
        'model'=>$model,
        'name' => 'scheduled_time',
        'options'=> array(
            'dateFormat' =>'yy-mm-dd',
            'altFormat' =>'yy-mm-dd',
        ),
    ));
    ?>
    <?php echo $form->error($model,'scheduled_time'); ?>
</div>

Remove the fields output, started, and completed from the form. These fields will be updated by the job running mechanism.

The resulting form should look like the following screenshot:

  1. Set the default value of active on a new scheduled job to true by adding the following line to the Create action in the Scheduled Job Controller in Source Files | protected | controllers | ScheduledJobController:
// set the default value of active to true
$model->active = true;
$this->render('create',array(
    'model'=>$model,
));
  1. Remove the link for Manage ScheduledJob from Source Files | protected | views | scheduledJob | create.php and from Source Files | protected | views | scheduledJob | update.php.
  2. Change the scheduled job view to show valid actions. In Source Files | protected | views | scheduledJob | view. php remove the link for Manage ScheduledJob.

Change the breadcrumb to show the job name.

$this->breadcrumbs=array(
    'Scheduled Jobs'=>array('index'),
    $model->job->name,
);

Also, change the header of the page to show the job name.

<h1>View Scheduled Job <?php echo $model->job->name; ?></h1>

Finally, replace job_id with the job name and move it to the top of the attribute list, and replace active with a string value of true or false.

'attributes'=>array(
    array (
        'name' => 'job_name',
        'header' => 'Job',
        'value' => $model->job->name,
        ),
    'params',
    'output',
    'scheduled_time',
    'started',
    'completed',
    array (
        'name' => 'active',
        'header' => 'Active',
        'value' => $model->active ? 'true' : 'false',
    ),
),

The finished screen should look like the following:

  1. Finally, we will streamline the scheduled job index to show a few fields that we care about. Edit Source Files | protected | views | scheduledJob | index.php and remove the id, params, and output fields from the index view grid. Replace the job_id field with the job name.
array (
    'name' => 'job_name',
    'header' => 'Job',
    'value' => '$data->job->name',
),

Replace the action field with a checkbox column.

array (
    'class'=>'CCheckBoxColumn',
    'id' => 'active',
    'header' => 'Active',
    'checked' => '$data->active',
    'selectableRows' => 0,
),

The resulting screen will look as follows:

Objective Complete - Mini Debriefing

In this section, we customized the scheduled job screens to make it easier to input and manage jobs as we test our job scheduling system.

Adding Job Processing

In this task, we will create a simple job consumption script. We will use cron to run the script every so often. When it runs, it will query for jobs that need to run in this time frame, queue them up, execute them, and record the results. We want the jobs to run within the context of Yii, because we want to:

  • Use the same database configuration as our web application
  • Take advantage of Yii's libraries, primarily the database access utilities

To do this, our job processing script will be written as a Yii command, like the RBAC command we used in Project 4, Level Up! Permission Levels.

Engage Thrusters

  1. Create a project directory named utils. Right-click on ch7 | Source Files | protected, select New | Folder, and enter utils. We will keep the job entry script and any other administrative scripts we create, in this directory.
  2. In the utils directory, create a custom Yii entry script named job_entry.php.
<?php
  // change the following paths if necessary 
  $yii='/opt/lampp/htdocs/yii/framework/yii.php';
  $config=dirname(__FILE__).'/../config/main.php';
 
  // remove the following lines when in production mode 
  defined('YII_DEBUG') or define('YII_DEBUG',true); 
  // specify how many levels of call stack should be //shown in each log message 
  defined('YII_TRACE_LEVEL') or define('YII_TRACE_LEVEL',0); 
  require_once($yii); 
  $app = Yii::createConsoleApplication($config)->run();
?>

Our custom entry script allows us to execute console commands within the context of our Yii application. It differs from the project's yiic.php, for example, in that it uses the main application configuration file, and has a separate debugging and trace level.

We set the YII_TRACE_LEVEL variable in job_entry.php to 0, so that the logging output we configure later will only report our log statements, and no additional trace lines of context.

  1. In the commands directory, create a file named JobProcessorCommand.php. Right-click on ch7 | Source Files | protected | commands, select New | PHP File, and enter JobProcessorCommand. This will be our job consumption script, and we will use the job_entry script to run it from the command line within our Yii context. Paste this skeleton command to initialize the job processor:
<?php
  class JobProcessorCommand extends CConsoleCommand
  {
        private function getJobs() {
            return ScheduledJob::model()->findAll();
        }
 
        public function run($args)
        {
            $jobs = $this->getJobs();
            foreach ($jobs as $job) {
                echo "Running " . $job->job->name . " scheduled for " . $job->scheduled_time . "\n";
            }
        }
  }
 
?>

For now, to demonstrate how job processing will run in the context of Yii, the script prints a list of jobs. Try running it by running the following commands from a terminal window:

cd ~/projects/ch7/protected/utils/
php job_entry.php jobprocessor

The script will produce a list of scheduled jobs that look like the following:

  1. Next, we want to narrow down the list of jobs to just active jobs scheduled for the current time. To help run this query, we will add named scopes to the ScheduledJob model. Named scopes provide a named query criteria that can be applied to an active record query. Edit Source Files | protected | models | ScheduledJob.php and add the following function:
public function scopes()
{
    return array(
        'active' => array(
            'condition' => 'active=1 AND completed IS NULL',
        ),
        'current' => array(
            'condition' => 'scheduled_time < now()',
        ),
    );
}
  1. Apply the active scope in our job processor by changing the scheduled job query to use the active scope.
private function getJobs() {
    return ScheduledJob::model()->active()->findAll();
}

When you run the jobprocessor command, the output shows only the jobs marked active and not yet complete.

  1. Add the current scope to the scheduled job query in the job processor.
private function getJobs() {
    return ScheduledJob::model()->active()->current()->findAll();
}

Now, the jobprocessor command output shows only active jobs, not yet marked complete, scheduled for the current time or earlier.

lomeara@YiiBook:~/projects/yiibook/Chapter 7/project_files/protected/utils$ 
php job_entry.php jobprocessor
Running Send Bday Wishlist Email scheduled for 2012-08-12 21:52:47
  1. At this point, we will configure logging, so that we can capture information as our jobs are run. Add a log route array for this configuration in ch7 | Source Files | protected | config | main.php. The array will contain a value for categories, so that we only apply this route to log entries for the jobprocessor category. It will also contain a value for fileName, to write the job log output to its own file job.log, separate from the web application output application.log. We set the level to Info, Error, and Warning so that all of these levels are written to the file. Our logging statements will be Info level.
'log'=>array(
    'class'=>'CLogRouter',
    'routes'=>array(
        array(
            'class'=>'CFileLogRoute',
            //'levels'=>'trace, error, warning',
            'levels'=>'error, warning',
        ),
        array(
            'class'=>'CFileLogRoute',
            'levels'=>'info, error, warning',
            'logFile'=>'job.log',
            'categories'=>'jobprocessor',
        ),
  1. Now change the run command in ch7 | Source Files | protected | commands | JobProcessorCommand.php to run the action specified in the job. We will supply the action with the given parameters. We will also write a log entry recording the job run, and store the json-encoded output and time started and completed. Note, this will temporarily break the command until we make some more changes.
public function run($args)
{
    $jobs = $this->getJobs();
    foreach ($jobs as $job) {
        Yii::log("Running - Job [" . $job->job->name .
            "] Action [" . $job->job->action .
            "] Parameters [" . $job->params . 
            "] scheduled for " . $job->scheduled_time,
            'info', 'jobprocessor');
        $name = $job->job->action;
        $job->started = new CDbExpression('NOW()');
        $job->save();
        $job->output = json_encode($this->$name($job->params));
        $job->completed = new CDbExpression('NOW()');
        $job->save();
    }
}
  1. To complete the job processing system, set up system automation to run the job processor every so often.

For example, on Unix, you could use crontab and schedule job processing once a night. Use the following command to open your user crontab for editing:

crontab -e

Select an editor, for example vim, and add the following line:

15 2 * * * php ~/projects/protected/utils/job_entry.php jobprocessor

Cron will run the job processor and subsequently all of the jobs scheduled for the previous day each night at 2:15 a.m.

You can learn more about crontab online and try out different configurations for running your job processor at different times or more frequently.

Objective Complete - Mini Debriefing

We have created a script to run our jobs within the context of our web app. It allows us to use our own project models and libraries in our job functions. We created a log configuration just for our job processor. Alternately, we could have created another config script for the job processor and passed that in job_entry.php. We demonstrated one way to configure our server to run our jobs every night.

Classified Intel

If you schedule many jobs, you may run into problems processing them; either because a job is very resource-intensive or many jobs are configured to run or some combination of these situations. To handle more job processing, you may want to look into adding multi-threaded support to your job processing. There are several ways to do this. One way to do this is to run them through the web server. We chose not to do this, because we did not want to permit our jobs to be run directly from the web. If you want to take this route, the jobs would either be web-accessible, or you could implement a token-based system to prevent unauthorized access to the jobs. See Project 5, Service Please – Integrating Service Data, for examples of using tokens and a discussion of secure token generation.

Creating and Registering a Job

The first job we will create will simply demonstrate how to create and call a job. It will not take parameters as input. It will not produce output. It will just run and send e-mails to the users of our system.

Prepare for Lift Off

To demonstrate this function, we have added an e-mail field to the user table. The e-mail field is roughly supported in the interface. You may want to expand on the support with validation and search features. For now, the create and update user screens will allow a user to view and edit the field.

The job that we are about to create will send an e-mail to all of your users. In order to send e-mails, you must have a mail server configured on your system.

If you do have a configured e-mail server, or you do not know whether or not you do, be careful about the e-mail addresses that are configured for your users. You may want to reduce the number of users in your database, and change any live e-mail addresses to fake e-mail addresses. @email.com is a good test e-mail domain to use.

If you do not have a configured e-mail server, you can still run the job function and verify the output in the job log file. We will create a reporting job later with more exciting output.

Engage Thrusters

  1. You should have only one active, current job entry, Send Bday Wishlist Email, from the default schema data. If that is not the case, update the information in the scheduled job table to match, so that your jobs run correctly in the following steps. You can do this quickly by dropping the table, recreating it, and reloading the original job schema file Source Files | protected | data | jobs.sql.
  2. The job we are creating will e-mail our wishlist to our friends. We want to use a list of users that does not include our account (admin), so let's add a scope to the user model.
public function scopes()
{
    return array(
        'not_admin' => array(
            'condition' => "username!='admin'",
        ),
    );
}
  1. Finally, create the job action to run. Add the following private function to Source Files | protected | commands | JobProcessorCommand.php:
private function SendWishlist() {
   // prepare the email message
   $subject = "Some Gift Ideas";
   $headers = 'From: My CBDB Admin admin@mycbdb.com' . "\r\n" .
   'Reply-To: My CBDB Admin admin@mycbdb.com' . "\r\n" .
   'X-Mailer: PHP/' . phpversion();
   $email = "Here are a few of my gift wishes:\n";
   // build the body of the email
   $wishes = Wish::model()->findAll();
   foreach ($wishes as $w) {
     $email .="\t" . $w->title . "\n";
   }
 
   $email .="Please come to my website to see more about " .
   "my collection and play some games.";
   Yii::log("My wishlist email message is [" . $email. "]",
   'info', 'jobprocessor');
 
   $wishgivers = User::model()->not_admin()->findAll();
   foreach ($wishgivers as $wg) {
     Yii::log("Sending wishlist to " . $wg->username,
     'info', 'jobprocessor');
     //mail($email, $subject, $body, $headers);
   }
}

We have commented out the actual command to send the e-mail in our job. When you run it, the only output will be log entries that are created. If you have a mail server set up and are comfortable sending an e-mail to all of your users, you can uncomment it and run it.

  1. To run this job, you can schedule the Send Bday Wishlist Email job for some time soon and change the time your job processor will run. Alternatively, you can run the job manually with the following command:
php ~/projects/ch7/protected/utils/job_entry.php jobprocessor

If you want to run the job repeatedly, you will need to reset the completed timestamp in the scheduled job record to NULL after each run. We did this by executing a MySQL command to reset the value. Alternately, you could add a function to your interface to support repeated testing.

  1. Check the log output in Source Files | protected | runtime | job.log to confirm that the job ran successfully. It should look something like the following:
2012/08/15 03:45:01 [info] [jobprocessor] Running - Job [Send Bday Wishlist Email] Action [SendWishlist] scheduled for 2012-08-12 21:52:47
2012/08/15 03:45:01 [info] [jobprocessor] My wishlist email message is [Here are a few of my gift wishes:
  Moebius' Airtight Garage Vol.1
  The Squiddy Avenger
  another great title
Please come to my website to see more about my collection and play some games.]
2012/08/15 03:45:01 [info] [jobprocessor] Sending wishlist to borrower
2012/08/15 03:45:01 [info] [jobprocessor] Sending wishlist to afriend
2012/08/15 03:45:01 [info] [jobprocessor] Sending wishlist to twg
2012/08/15 03:45:01 [info] [jobprocessor] Sending wishlist to tcreate

Objective Complete - Mini Debriefing

In this section we created a simple e-mail sending job, scheduled, and tested it.

Classified Intel

When you want to run a job many times to debug it, remember to reset the scheduled job entry in the database. If you do not, the job processor will see the scheduled job as already run and will not pick it up and run it. All you need to do is set completed to null, but you could set completed, started, and output to null. The following is a MySQL command you can use to set those values to null for all scheduled jobs:

UPDATE scheduled_job SET completed=null,started=null, output=null;

Creating a Graphical Report

Now we will write a job that generates JSON report data that can be displayed graphically. We chose to use the Flot JavaScript library (http://code.google.com/p/flot/) to present our report data, but you could write a report to output any reporting format you like. You could even generate an Excel spreadsheet.

Our report will produce a bar graph of our books by grade. We will set up Flot in the next task. For now, we will concentrate on writing a query and storing the JSON output.

Engage Thrusters

  1. In the web app, edit the Generate a Report scheduled job, and change the scheduled time. You can use the Now button to quickly change the value to a time that will soon have past. Make sure that the active field is set to 1.
  2. Open Source Files | protected | commands | JobProcessorCommand.php and create a private function named RunReport.
private function RunReport() {
}
  1. Add a query to get the number of books by grade.
$criteria= new CDbCriteria();
$criteria = array(
    'select' => 'count(grade_id) as num_grade, grade_id',
    'with' => array( 'grade' ),
    'group' => 'grade_id',
);
$books = Book::model()->findAll($criteria);
  1. We will not be able to access the num_grade value until we add that field to the Book model (Source Files | protected | models | Book.php).
class Book extends CActiveRecord
{
    public $borrower_fullname = '';
    public $borrower_fname;
    public $borrower_lname;
    public $num_grade;
  1. Add the following code after the query in the RunReport function to initialize the Flot report.
// initialize report
$report = array(
    'data'=> array (
        array(
            'label'=> 'Comic Books by Grade',
            'data'=>array(),
            'bars'=>array(
                'show'=>true,
                'align'=>'center',
            ),
        ),
    ),
    'options'=>array(
        'legend'=>array(
            'show'=>false,
        ),
    ),
    'htmlOptions'=>array(
        'style'=>'width:200px;height:200px;'
    )
);
  1. After the report initialization, add the following for loop to iterate over the query results and add them to the report as data points. Return the result.
foreach ($books as $book) {
    $report['data'][0]['data'][] = array($book->grade_id,$book->num_grade);
    $report['options']['xaxis']['ticks'][] = array($book->grade_id,$book->grade->name);
}
 
return $report;
  1. Run the report from the command line to capture the data.

Objective Complete - Mini Debriefing

For now, we have a reporting job that silently runs and collects the current grading status of our comic book collection. In the next section, we will display the results.

Displaying Graphical Report Output

In the previous section, we collected and prepared data. In this section, we simply have to add a graphing extension to display it. We chose to use the Flot extension because we liked the look and ease of use of the extension.

Prepare for Lift Off

  1. Download the Yii extension EFlot from http://www.yiiframework.com/extension/flot/.
cd ~/Downloads
wget http://www.yiiframework.com/extension/flot/files/EFlot.zip
  1. Unzip the package in your project's extensions directory.
cd ~/projects/ch7/protected/extensions/
unzip ~/Downloads/EFlot.zip
  1. Add the following entry to the import array in ch7 | Source Files | protected | config | main.php.
 'ext.EFlot.*'

Engage Thrusters

  1. Update the scheduled job view, Source Files | protected | views | scheduledJob | view.php, to check for output data. If output is not null, pass the JSON-decoded data to the EFlot widget.
<?php
    if ($model->output != null) {
        $this->widget('application.extensions.EFlot.EFlotGraphWidget',
            json_decode($model->output, true)
        );
    }
?>
  1. Open the scheduled job view for the Generate a Report job to see the results.

Objective Complete - Mini Debriefing

Our graph applies a minimal number of the features available. You may want to explore the options you can give Flot to produce different labels and charts. You also have the capability to add more jobs that query our data and prepare graphical reports.

Mission Accomplished

In this project, we have built a system to input, schedule, and process jobs. We have included support for running reporting jobs that generate graphical output and viewing their results.

You Ready to go Gung HO? A Hotshot Challenge

Here are some ideas to extend on the work from this chapter:

  • Support priority in the job queue – both recording the priority for a scheduled job and applying the priority when choosing which jobs to run.
  • Add support for scheduling recurring jobs.
  • Replace job processing cron script with a full-time job processing daemon.
  • Add Ajax to the job-scheduling grid to set jobs as active/inactive.
  • Create more reporting jobs to try out the different types of reports you can create.
  • Update the output format to include flags that indicate what type of data is stored and how it should be generated. Use this to support the EFlot format and some other formats, for example simple text output.
评论 X

      友荐云推荐
      Copyright 2011-2014. YiiBook.com