Pro Blog integration

Permalink Browser Info Environment
The top concrete5 blogging addon Pro Blog now integrates with Magic Data.

http://www.concrete5.org/marketplace/addons/problog/...

====================================
Update - 10 January 2014
For anyone coming to this thread, the posts below lead through a process of building and filtering a Magic Data expression for Uber List, finding out that the filter process was a not fast enough when there was a very big list to filter, and adding a couple of new symbols to provide lightning fast listing and filtering for such cases.

On the way there were a few catches, like text strings not matching because the search indexing string was in the database without being trimmed.

Type: Discussion
Status: Archived
JohntheFish
View Replies: View Best Answer
FatTony1952 replied on at Permalink Reply
FatTony1952
So I'm just getting around to really finding out what this Magic Data is all about because I've run into a situation with the Pro Blog addon that this might be useful...if it can do what I'd like it to do.

I have a client with a site where users can purchase a membership. That membership purchase creates a user profile page for the member. The members submit news to the site admin to post into the blog section. Each blog entry will have a category attribute tied to it with the same name as the Business Name user attribute. I'd like to sort the blog posts by the users' categories on their profile pages.

For instance, Sam's Butcher Shop's profile page would show only blog posts with the category, "Sam's Butcher Shop".

I've read a bit of the documentation on this and it seems like it may work. Thoughts?
JohntheFish replied on at Permalink Reply
JohntheFish
There are 2 parts to this.

1. Make the list of posts. Magic data can list pages of a type, then filter that list by matching the business name attribute to a user profile attribute (or even a user name).

2. Format the list for display. Here you have a number of options:

2a. Build the list output as html directly using Magic Data.

2b. Use Uber List (which also incorporates 1)

2c. Build a template for the blog list block that gets its list of posts from a magic data expression.

If I were doing this, 2b is the solution I would use. Its the sort of combined list and formatting requirement that Uber List excels at.
FatTony1952 replied on at Permalink Reply
FatTony1952
Thanks. I just got all the necessary add-ons to get started on this.
FatTony1952 replied on at Permalink Reply
FatTony1952
Okay, so I have my list created

[code]
'blog' AS_PAGE LIST_PAGES 10 FILTER_LIST ATTRIBUTE business_name END_FILTER
[code]

I need to make the list filter on the business_name being equal to the blog category name. I didn't see a symbol for the blog category.
JohntheFish replied on at Permalink Best Answer Reply
JohntheFish
What you need to do here is to compare 2 attributes, the blog category of the blog entry page and the business of the user profile.

As a development strategy, have a look at
http://www.c5magic.co.uk/add-ons/magic-data/users-groups-examples/...
This is not your requirement, but it shows the process of building and testing parts before adding them together.

The Magic Data below is only briefly tested, so you should adopt the same strategy and test the individual parts in the symbol tester before assembling them in the uber list block.

PROFILE ATTRIBUTE 'business_name' SAVE 'bus_nm'
'blog' AS_PAGE LIST_PAGES 10 
APPLY_EACH
  LIST_PAGES 12 END_ON_NULL REDUCE_LIST MERGE_INTO 'months'
END_APPLY_EACH
'months' RESTORE
APPLY_EACH
  LIST_PAGES 100 END_ON_NULL REDUCE_LIST MERGE_INTO 'posts'
END_APPLY_EACH
'posts' RESTORE
FILTER_LIST 
  ATTRIBUTE 'blog_category' EQ ( 'bus_nm' RETRIEVE )
END_FILTER


The first line retrieves the Business Name attribute from the profiled user and saves it for convenience. When testing and developing in the symbol tester, you may want to swap PROFILE to USER and then back again when ready because PROFILE only works reliably when it is used on a /profile/ page.
USER ATTRIBUTE 'business_name' SAVE 'bus_nm'


Line 2 you already have. But ProBlog has a hierarchical structure and LIST_PAGES only list those immediately below, so you will only get lists of years, not of posts. The solution is the further lists to list down within each, first to months, then to individual posts.

END_ON_NULL - quits early if a list is empty
REDUCE_LIST - removes empty and duplicate items from a list
MERGE_INTO - accumulates lists into a memory ('months' or 'posts').

The key part is the filter condition, where we take the 'blog_category' attribute of each listed post and compare it to the previously saved 'bus_nm'. Note the parenthesis to ensure we are comparing to the actual retrieved value and not the text of the symbol.

As noted, you should develop and test each fragment then assemble. For real use you may want more than 10 at each level of the list, such as allowing for 12 months in a year. You may also need to check that the attribute handles are correct.
FatTony1952 replied on at Permalink Reply
FatTony1952
Okay, I think I'm getting the hang of this thing now. Pretty powerful and can solve a LOT of issues I've run into recently.

I do occasionally get this error:

An unexpected error occurred. Unable to get permission key for write< Back to Home

Not sure what it means though.
JohntheFish replied on at Permalink Reply
JohntheFish
The permission key error is reported when the ajax connection from the symbol tester to the server does not have the correct c5 validation key. This could happen if, for example, you spend so long in the symbol tester that the validation key becomes out of date, or if you use the browser back button to return to the symbol tester from another page and the symbol tester page is served from the browser cache.

This is a general limitation of the c5 validation key mechanism.
FatTony1952 replied on at Permalink Reply
FatTony1952
I picked this apart to test all the pieces and they all seem to work. However, I can only get one of the member's posts to filter.

USER 95 ATTRIBUTE 'business_name' SAVE 'bus_nm'
'blog' AS_PAGE LIST_PAGES 10 
APPLY_EACH
  LIST_PAGES 12 END_ON_NULL REDUCE_LIST MERGE_INTO 'months'
END_APPLY_EACH
'months' RESTORE
APPLY_EACH
  LIST_PAGES 100 END_ON_NULL REDUCE_LIST MERGE_INTO 'posts'
END_APPLY_EACH
'posts' RESTORE
FILTER_LIST 
  ATTRIBUTE 'blog_category' EQ ( 'bus_nm' RETRIEVE )
END_FILTER


Any other user ID I place in there always retrieves a null result.

I tested a direct pull to make sure I had the user ID correct and was able to pull the business name from the user's attributes as well as pull the blog posts with the same category name.
JohntheFish replied on at Permalink Reply
JohntheFish
In general, the first data value any Magic Data symbol takes is the result of the preceding symbols. So the general pattern is
data1 SYMBOL param1 param2


So to specify a user by ID you do
95 AS_USER ATTRIBUTE 'business_name'


Or in the symbol tester, use the link below the symbol entry field to set the context for the 'current user' and USER will return that user ID.
USER ATTRIBUTE 'business_name'


Though remember that on a profile page to get a user according to the profile you need to edit that to:
PROFILE ATTRIBUTE 'business_name'



With Uber List now released I suspect multiple level lists could become a common requirement, so I will look into extending the capability of LIST_PAGES for multiple levels or creating a new multi-level page listing symbol.
JohntheFish replied on at Permalink Reply
JohntheFish
V1.15 of Magic Data Symbols extends LIST_PAGES to accept an optional number of levels, so the above can now be simplified to:
PROFILE ATTRIBUTE 'business_name' SAVE 'bus_nm'
'blog' AS_PAGE LIST_PAGES 100 -3
FILTER_LIST 
  ATTRIBUTE 'blog_category' EQ ( 'bus_nm' RETRIEVE )
END_FILTER
FatTony1952 replied on at Permalink Reply
FatTony1952
I'm wondering if the issue has more to do with the blog categories than the Uber List add-on. I can still only pull the one blog post (of the dozen or so that I've tested against).
JohntheFish replied on at Permalink Reply
JohntheFish
You can check the categories for pages by looking at the page attributes, either from the page in edit>properties, or from the sitemap.

There may also be some bulk attribute viewing addons similar to the bulk seo, but for attributes.

EDIT:
You can use Page Search and under the advanced options, add the blog_category attribute to the columns shown.
JohntheFish replied on at Permalink Reply
JohntheFish
If adding pro-blog was a recent move, there could also be confusion between blog pages inherited from the previous core blog and blog pages added since pro-blog.
JohntheFish replied on at Permalink Reply
JohntheFish
You can create a quick list of pages : category in the symbol tester using:
'blog' AS_PAGE LIST_PAGES 100 -3
APPLY_EACH
  SAVE m1
  PAGE_NAME . ' : ' . ( m1 RESTORE ATTRIBUTE 'blog_category' )
END_APPLY_EACH
HTML_OL
FatTony1952 replied on at Permalink Reply
FatTony1952
Just got an error: MySQL 1030 error after the update.
JohntheFish replied on at Permalink Reply
JohntheFish
According to stackoverflow, a mySQL 1030 error can occur because either your database has a corrupted table or has run out of working space.

http://stackoverflow.com/questions/10631387/1030-got-error-28-from-...
http://stackoverflow.com/questions/4067461/mysql-error-code-1030got...
FatTony1952 replied on at Permalink Reply
FatTony1952
I uninstalled and reinstalled and everything seems to be fine now.

I've posted in the Pro Blog forum about the post categories. I'm wondering if there is a way to re-index the blog posts somehow. I see in the results that there are lots of posts within categories, but some of those profile pages still show up with no results.
JohntheFish replied on at Permalink Reply
JohntheFish
The match in 'EQ' is a simple text match, so things like different white space in a user's business_name attribute, different capitalisation etc could be throwing things off. You can apply TRIM to the business_name and then use EQC for case-insensitive matching to get round this.

Another possibility could be that you need to give the LIST_PAGES a big enough limit to list all the possible pages that are relevant before you then begin filtering them. So while there may only be a few results of interest, you need to start by listing all blog entries before reducing by the attribute match to just those results.

As a check, in the symbol tester try:
'blog' AS_PAGE LIST_PAGES 100 -3
FILTER_LIST
  ATTRIBUTE 'blog_category'
END_FILTER_LIST
APPLY_EACH
  SAVE m1
  PAGE_LINK . ' : ' . ( m1 RESTORE ATTRIBUTE 'blog_category' )
END_APPLY_EACH
HTML_OL

This will list all posts that have a blog_category attribute, whatever its value is.

The equivalent for users would be
'Business Users Group Name' AS_GROUP LIST_USERS 100
FILTER_LIST
  ATTRIBUTE 'business_name'
END_FILTER_LIST
APPLY_EACH
  SAVE m2
  USERNAME . ' : ' . ( m2 RESTORE ATTRIBUTE 'business_name' )
END_APPLY_EACH
HTML_OL


As a diagnostic, the above 2 expressions will give you 2 lists:
a) A list of blog posts with categories
b) A list of users with a business name attribute

Between them, the above should tell you if you have the basic data necessary to extract the lists.
JohntheFish replied on at Permalink Reply
JohntheFish
The listing expression with TRIM and EQC:

PROFILE ATTRIBUTE 'business_name' TRIM SAVE 'bus_nm'
'blog' AS_PAGE LIST_PAGES 100 -3
FILTER_LIST 
  ATTRIBUTE 'blog_category' TRIM EQC ( 'bus_nm' RETRIEVE )
END_FILTER
FatTony1952 replied on at Permalink Reply
FatTony1952
This thing is a LIFE SAVER! I'm finally at a point where I have some time to really fiddle with it and it's definitely magic!

I have noticed there is a bit of a lag in some of the queries before they post. Is there a way to speed it up?
JohntheFish replied on at Permalink Reply
JohntheFish
In the symbol tester, much of the time for complex expressions, especially where loops are involved, is the diagnostic trace.

The overhead of the diagnostic trace is not applicable when Magic Data expressions are used in the wild (unless you opt to log it).

I have another Magic Data extension in the PRB 'Magic Data Developer' that is a compendium of developer resources. Amongst the further symbols are some for benchmarking and for disabling the trace in the symbol tester(so the benchmarks are accurate!).

With Uber List, the MD expression to create the list is actually evaluated pretty fast, fractions of a second for some quite complex lists, and the built in cache can make that almost instant.

Where it can slow down is on the complexity of the stack rendered for each list item. If that is kept as simple as a typical page list - title and description - then speed is only slightly slower than a traditional page list block. If the stack is complex with lots of content and much formatting, time to show then extends proportional to the number of items in the list. On my roadmap is lazy/background loading of listed items after the first page - as is used in the best image gallery addons.
FatTony1952 replied on at Permalink Reply
FatTony1952
I have the logging off on the page list and my stack is just a simple 'Title/Description/Link'.

Can you take a look and see what you think? I'm using the MD/UL on the individual user page to pull in the blog. You can click through from the link below to see the load time I'm referring to.

http://jrbrinkman.com/our-members/...
JohntheFish replied on at Permalink Reply
JohntheFish
I like the site. Clean, attractive and easy to follow.

The delay is certainly nothing to do with the rendering of individual Uber List items, because the links I have clicked had only 0 or 1 items to show!
( 1 item for The Rocket Inn:http://jrbrinkman.com/index.php/profile/61/... )

So the question is whether the time taken stems from evaluating the list expression or from the page itself. The time I am seeing is similar to that of showing an individual item page in the news (though individual blog items render quickly), so it could be a page/attribute complexity issue within that area of the site and not something Magic Data or Uber List can control.

Are there any entries in the concrete5 log relating to this?

Some tests to help narrow the problem down:

1. What is the speed like showing profile pages without any Uber List block? Perhaps with a content block or stack with some dummy content in its place.

2. What happens if you run similar Magic Data to create a list directly in an html block on a test page (ie not in a profile page and not within Uber List)? Such as:
SET '61' AS_USER ATTRIBUTE 'business_name' TRIM SAVE 'bus_nm'
'blog' AS_PAGE LIST_PAGES 100 -3
FILTER_LIST 
  ATTRIBUTE 'blog_category' TRIM EQC ( 'bus_nm' RETRIEVE )
END_FILTER
IMPLODE

(You will need to assign the html block a magic data template).

3. What is the speed like if you run the same Magic Data from the dashboard symbol tester?

The above 3 tests will confirm if this is related to profile pages generally or to showing the list in profile pages or to the list generating expression directly.

Also, please double check that 'Enable logging of token symbol evaluation' is not checked in the dashboard page: 'Magic Data - Symbols Settings'.
FatTony1952 replied on at Permalink Reply
FatTony1952
I've tested a few different things. I tested the profile page without the Uber List block and it responds quickly.

I also tried with the Magic Data pulling in something simple, i.e. the page name and it still lags just a little, just not as much as when I pull in the page name, truncated content and a link. When I run the query from the dashboard tester, it has a bit of a lag, again, not as much as the actual profile page, but a little bit. If I run the test on just one record, then it's very snappy. I think there is something to be said about the page attributes possibly slowing the page down a bit also.

I haven't had the logging enabled from the get-go. Figured it would start to pile up entries as I tested, etc.
JohntheFish replied on at Permalink Reply
JohntheFish
Am I correct in summarising that the significant lag is only occuring when all these things come together: profile page, complex list to filter, complex list item content ?

Is the jrbrinkman.com site a development site? Could you give me admin access (by PM, not on this open forum) to have a look and run some experiments myself?

If so, I would like to install the MD developer extension so I can take some benchmarks (I would send you a zip of the package for that to install by ftp)
FatTony1952 replied on at Permalink Reply
FatTony1952
Yes, the lag happens when everything comes together and yes, it is a development site.

I'm PMing you the login info.
JohntheFish replied on at Permalink Reply
JohntheFish
Version 1.1. of Uber List now includes a symbol LIST_ALL_PAGES_WITH_FILTER which integrates much of the above into a single symbol with considerable optimisation.

Before updating Uber List, you must first update Magic Data and Magic Data Symbols because Uber List requires the latest versions of both.

'blog' LIST_ALL_PAGES_WITH_FILTER 'blog_category' ( PROFILE ATTRIBUTE 'business_name' TRIM ) 100


The first parameter is the attribute to filter on. Second parameter - here expanded within the parenthesis - is the value to match against. Third parameter is the maximum number of pages to list.

Because the filtering is now integrated within the listing, the maximum number only needs to be big enough for the posts by that business, not for all possible posts.
FatTony1952 replied on at Permalink Reply
FatTony1952
Cool. Does this eliminate the need to check down to a certain level of pages and eliminating the
AS_PAGE LIST_PAGES 100 -3
JohntheFish replied on at Permalink Reply
JohntheFish
Yes, the core method used searches all descendants, whatever the level, so there is no level parameter for this symbol.
JohntheFish replied on at Permalink Reply
JohntheFish
You may want to tidy up slightly my expression above. The entire Uber List selection expression can be as simple as:
SET 'blog' AS_PAGE 
LIST_ALL_PAGES_WITH_FILTER 
'blog_category' 
( PROFILE ATTRIBUTE 'business_name' TRIM ) 
100
FatTony1952 replied on at Permalink Reply
FatTony1952
I'm not getting it to list any posts.

For instance, this page should have several posts listed -http://jrbrinkman.com/index.php/profile/152/...
JohntheFish replied on at Permalink Reply
JohntheFish
The symbol uses the c5 search index, so you may have to tick the checkbox so that the attribute is searchable and then run the index search engine job.
FatTony1952 replied on at Permalink Reply
FatTony1952
I have done that and still no results. I actually set up a cron job to run the search index and to get rid of old pages and it runs daily.
JohntheFish replied on at Permalink Reply
JohntheFish
Is it OK for me to login again and have a look?
FatTony1952 replied on at Permalink Reply
FatTony1952
Absolutely. I re-ran the crons from the Dashboard to ensure a fresh update.
JohntheFish replied on at Permalink Reply
JohntheFish
While I am poking about, please have a look at the database with phpMyAdmin. When the search engine is indexed for this attribute the table:
CollectionSearchIndexAttributes

Will have a column:
ak_blog_category

In that column should be the names of the various businesses marked against each blog page indexed.
FatTony1952 replied on at Permalink Reply
FatTony1952
Those all look correct
JohntheFish replied on at Permalink Reply
JohntheFish
I took a dump of the database and examined it in a text editor. I can now see why it is not matching. It is essentially the same problem as encountered earlier that was solved by trimming both the blog_category attribute value and the comparison value.

The search index values for the blog_category attributes are entered in the database with a new line before and after the value (not sure if this will show here), for example:
\nHalf Pint Concepts\n


Within LIST_ALL_PAGES_WITH_FILTER, its only possible to trim the comparison value, not the blog_category attribute value, hence the comparison is always failing.

It may be possible to get round this with some adjustments of the SQL the core search class generates.
JohntheFish replied on at Permalink Reply 1 Attachment
JohntheFish
I have not been able to get a general purpose comparison with full whitespace trim working. TRIM in mySQL is a particularly limited implementation and various work-rounds soon become very cumbersome.

As an alternative solution, I have attached a revised symbol LIST_ALL_PAGES_WITH_FILTER_CONTAINING which implements a containing match. To use it, please FTP and rename the attached file into:

/packages/jl_uber_list/models/magic_data/symbols/list_all_pages_with_filter_containing.php

I have hacked one of my databases to have data with a similar \n issue to yours and it tests out OK for me.

SET 'blog' AS_PAGE 
LIST_ALL_PAGES_WITH_FILTER_CONTAINING
'blog_category' 
( PROFILE ATTRIBUTE 'business_name' TRIM ) 
100


Because it implements a containing match, it could return false positives if you have similar business names where one name contains another. If you find that a problem, you will need to follow the above with the FILTER_LIST ... END_FILTER used before to remove false positives from the (short) result list.

Let me know how you get on and I will add the symbol to Uber List and release another upgrade.
FatTony1952 replied on at Permalink Reply
FatTony1952
I'd love to give you an update on this, but after replacing the file and editing the profile page, I can't connect to the site anymore.

I'll have to get back to you when I figure this issue out.
JohntheFish replied on at Permalink Reply
JohntheFish
I can't even get a ping response. Maybe restart apache.
FatTony1952 replied on at Permalink Reply
FatTony1952
It's a Bluehost issue.
FatTony1952 replied on at Permalink Reply
FatTony1952
Okay, now it gives me a link that says "Database XML"
JohntheFish replied on at Permalink Reply
JohntheFish
I have logged in and:

- Created a working list example in the dashboard symbol tester. To be safe, I also added the secondary filter I mentioned above to remove any false positive results.
http://jrbrinkman.com/index.php/dashboard/magic_data/symbols/...

- Put that list example into an Uber List block on a test page at
http://jrbrinkman.com/index.php?cID=1321...

- Then inserted the list condition part into the Uber List block on profile pages.
http://jrbrinkman.com/index.php/profile/22/...

On the profile pages I have visited so far, it appears to be both listing posts accurately and fast. However, I have only tested 10 or so pages.

The expression in the dashboard symbol tester is
SET "Burchell's White Hill Farmhouse Inn" TRIM SAVE 'bus_nm'
SET 'blog' AS_PAGE 
LIST_ALL_PAGES_WITH_FILTER_CONTAINING 'blog_category'  ( SET 'bus_nm' RETRIEVE ) 100
FILTER_LIST 
  ATTRIBUTE 'blog_category' TRIM EQC ( SET 'bus_nm' RETRIEVE )
END_FILTER


And the expression on the profile pages is:
PROFILE ATTRIBUTE 'business_name' TRIM SAVE 'bus_nm'
SET 'blog' AS_PAGE 
LIST_ALL_PAGES_WITH_FILTER_CONTAINING 'blog_category'  ( SET 'bus_nm' RETRIEVE ) 100
FILTER_LIST 
  ATTRIBUTE 'blog_category' TRIM EQC ( SET 'bus_nm' RETRIEVE )
END_FILTER
JohntheFish replied on at Permalink Reply
JohntheFish
There seems to be a mis-mathced post for Barb's Buttery Toffee
http://jrbrinkman.com/index.php/profile/172/...

Looking at the post, it appears it may have been wrongly categorised.
JohntheFish replied on at Permalink Reply
JohntheFish
The Burchell profiles is getting a narrow footer, it appears to be arising from a mis-matched div in a post/intro containing the text:

Rooted in Family
Normal 0 MicrosoftInternetExplorer4 more.

concrete5 Environment Information

Browser User-Agent String

Hide Post Content

This will replace the post content with the message: "Content has been removed by an Administrator"

Hide Content

Request Refund

You have not specified a license for this support ticket. You must have a valid license assigned to a support ticket to request a refund.