Installation Requirements

This addon requires that Magic Data is already installed. You should also install Magic Data Symbols.

If you want to create reports on eCommerce, you will also need Magic Data Commerce and may find Similar & Recently Purchased Products comes in useful.

Getting Started

Before you start, use the File Manager to upload an empty file of type .txt or .csv to use as the destination for Magic Job or Magic Queueable Job output.

In the dashboard, go to Dashboard > Magic Data > Magic Job or Dashboard > Magic Data > Magic Queueable Job

On the Output File tab, select the file you have just uploaded for the job output.

Magic Job

On the Job Expression tab, build an expression of Magic Data symbols to create the report. For complex reports you can accumulate elements of the reports into memories then pull them all together at the end.

The FORMAT symbol can be useful for putting the parts of a report together. Magic Job also provides the symbols ESCAPE_CSV to correctly escape elements for csv output as individual csv fields and NEWLINE to insert new lines into the output.

A simple example

List new blog pages added today. This adds rows to a text file listing new blog posts. Run the Magic Job just before midnight to record today's new posts.

TIME DATE 'Y-m-d H:i:s' SAVE "report_date"
00:01 TIME_TODAY SAVE "start_time"
  TIME_PUBLIC GE ("start_time" RESTORE )
IMPLODE_NICELY SAVE "todays_posts"
"{{report_date}} New blog post(s): {{todays_posts}}" FORMAT NEWLINE

Some explanation

1. Today's date, saved to a memory.

2. The start time of our report, saved to another memory

3. List the 10 most recent pages beneath the blog page

4,5,6. Filter the list to only keep pages published today

7. Give up now if there are no pages

8,9,10. Get the page name for each of todays posts

11. Join all the page names into a single string and save it to another memory

12. Format a line of the report

Each time the Magic Job is run, any pages added today beneath /blog will be listed and the list appended to the output file. So the best time to run the job is once per day late in the evening.

2013-10-03 23:51:46 New blog post(s): Magic Job for scheduled reports

To be absolutely accurate, a better development of this expression would be to work out yesterdays new posts and run the job early in the morning. Its only a few lines extra, involving a start time and an end time and an end test.

If you want to get really advanced, you can also use a Magic Data expression to select the output file ID.

Magic Queueable Job

Where a job is sufficiently big to run into server/php execution limits, you can breaki it up into slices of a Magic Queueable Job.

In a Magic Queueable Job, the actual job expression is split into two parts. 

List Items

The List Items expression returns a list of items that will be processed by the job, such as a list of pages, products or users.

This expression  is evaluated once at the start of the queueable job.

Apply expression to item

Magic Queueable Job then runs the Apply expression to item expression for each individual item listed by List Items.

The identity of the List Item being processed can be retrieved using the symbol QUEUEABLE_JOB_ITEM.

Job Queue and slices

Internally, when the job is started, the results of the List Items expression are individually added to a queue for the job. Then each job slice processed a number of items from the queue. This has a system default value of 10 in concrete5.6.3, but that value can be changed by entering another Magic Data expression in Queue Slice Size.

Output File

If you opt to use a Magic Data expression to identify the output file, this expression will be evaluated for each item in the job.

You could have one file for all job output, or if you want, you can have a different output file for each item in the job!

Setting up such a plethora of files could be an unmanageable exercise. However, Black Magic Data provides symbols for creating, deleting and clearing files that can make this easy.

Magic Snippet Job

A job to execute a Magic Data snippet named in the query parameter 'snippet' with input optionally provided by the query parameter 'previous'. Spaces should be substituted with '+' symbols as with any escaped URL parameter.



This job needs to be run from cron, from a terminal, directly entered in to the browser address bar, or similar URL fecthing method. It can't be run from the dashboard jobs page.

For example:

In the dashboard, create and save a snippet named 'Describe' containing the Magic Data


The job URL to get the abbreviated description for the home page would then be


Magic Flush and Fill Job

A special purpose variation of the Magic Queueable Job that flushes the cache and then fills pages according to a magic data list.

Like Magic Queueable Job, the list of pages to fill is provided by a Magic Data expression entered through the Magic Flush and Fill Job dashboard page.

This enables the job to automatically adapt to fill pages according to the expression evaluation.

Symbols and more

To see what other symbols are available you can find a live copy of the symbols list and a live symbols documentation page on my support site. The site also provides further examples and I like to regularly add more in response to requests from users.

If you require particularly complex reports, you should consider developing your report as one or more custom symbols. The ideal split is to use PHP to do the heavy processing, but use Magic Data to provide flexibility, so you don't need to re-write PHP code every time you need a minor adjustment of a report.

For example, in PHP you could write the 'pages added' example above as a single custom symbol that took the parent page and date as parameters.