This addon requires that Magic Data is already installed. You should also install Magic Data Symbols.
If you want to create reports on eCommerce, you will also need Magic Data Commerce and may find Similar & Recently Purchased Products comes in useful.
Before you start, use the File Manager to upload an empty file of type .txt or .csv to use as the destination for Magic Job or Magic Queueable Job output.
In the dashboard, go to Dashboard > Magic Data > Magic Job or Dashboard > Magic Data > Magic Queueable Job
On the Output File tab, select the file you have just uploaded for the job output.
On the Job Expression tab, build an expression of Magic Data symbols to create the report. For complex reports you can accumulate elements of the reports into memories then pull them all together at the end.
The FORMAT symbol can be useful for putting the parts of a report together. Magic Job also provides the symbols ESCAPE_CSV to correctly escape elements for csv output as individual csv fields and NEWLINE to insert new lines into the output.
List new blog pages added today. This adds rows to a text file listing new blog posts. Run the Magic Job just before midnight to record today's new posts.
TIME DATE 'Y-m-d H:i:s' SAVE "report_date"
00:01 TIME_TODAY SAVE "start_time"
'/blog' AS_PAGE LIST_RECENT_PAGES 10
FILTER_LIST
TIME_PUBLIC GE ("start_time" RESTORE )
END_FILTER
END_ON_NULL
APPLY_EACH
PAGE_NAME
END_APPLY_EACH
IMPLODE_NICELY SAVE "todays_posts"
"{{report_date}} New blog post(s): {{todays_posts}}" FORMAT NEWLINE
Some explanation
1. Today's date, saved to a memory.
2. The start time of our report, saved to another memory
3. List the 10 most recent pages beneath the blog page
4,5,6. Filter the list to only keep pages published today
7. Give up now if there are no pages
8,9,10. Get the page name for each of todays posts
11. Join all the page names into a single string and save it to another memory
12. Format a line of the report
Each time the Magic Job is run, any pages added today beneath /blog will be listed and the list appended to the output file. So the best time to run the job is once per day late in the evening.
2013-10-03 23:51:46 New blog post(s): Magic Job for scheduled reports
To be absolutely accurate, a better development of this expression would be to work out yesterdays new posts and run the job early in the morning. Its only a few lines extra, involving a start time and an end time and an end test.
If you want to get really advanced, you can also use a Magic Data expression to select the output file ID.
Where a job is sufficiently big to run into server/php execution limits, you can breaki it up into slices of a Magic Queueable Job.
In a Magic Queueable Job, the actual job expression is split into two parts.
List ItemsThe List Items expression returns a list of items that will be processed by the job, such as a list of pages, products or users.
This expression is evaluated once at the start of the queueable job.
Apply expression to item
Magic Queueable Job then runs the Apply expression to item expression for each individual item listed by List Items.
The identity of the List Item being processed can be retrieved using the symbol QUEUEABLE_JOB_ITEM.
Job Queue and slices
Internally, when the job is started, the results of the List Items expression are individually added to a queue for the job. Then each job slice processed a number of items from the queue. This has a system default value of 10 in concrete5.6.3, but that value can be changed by entering another Magic Data expression in Queue Slice Size.
Output File
If you opt to use a Magic Data expression to identify the output file, this expression will be evaluated for each item in the job.
You could have one file for all job output, or if you want, you can have a different output file for each item in the job!
Setting up such a plethora of files could be an unmanageable exercise. However, Black Magic Data provides symbols for creating, deleting and clearing files that can make this easy.
A job to execute a Magic Data snippet named in the query parameter 'snippet' with input optionally provided by the query parameter 'previous'. Spaces should be substituted with '+' symbols as with any escaped URL parameter.
ie:
[run-this-job-url]&snippet=your+snippet+name&previous=incoming+value+for+snippet
This job needs to be run from cron, from a terminal, directly entered in to the browser address bar, or similar URL fecthing method. It can't be run from the dashboard jobs page.
For example:
In the dashboard, create and save a snippet named 'Describe' containing the Magic Data
PAGE_DESCRIPTION TRUNCATE_TO_WORD 30
The job URL to get the abbreviated description for the home page would then be
[run-this-job-url]&snippet=describe&previous=1
A special purpose variation of the Magic Queueable Job that flushes the cache and then fills pages according to a magic data list.
Like Magic Queueable Job, the list of pages to fill is provided by a Magic Data expression entered through the Magic Flush and Fill Job dashboard page.
This enables the job to automatically adapt to fill pages according to the expression evaluation.
To see what other symbols are available you can find a live copy of the symbols list and a live symbols documentation page on my support site. The site also provides further examples and I like to regularly add more in response to requests from users.
If you require particularly complex reports, you should consider developing your report as one or more custom symbols. The ideal split is to use PHP to do the heavy processing, but use Magic Data to provide flexibility, so you don't need to re-write PHP code every time you need a minor adjustment of a report.
For example, in PHP you could write the 'pages added' example above as a single custom symbol that took the parent page and date as parameters.