Magic Job - Add users to groups
Permalink Browser Info Environment
Hi John,
I hope you're well.
Do you think it would be possible for be to use Magic Job to search through my users information and add them to a group depending on what text is found?
So, for instance here's what I'm thinking. I have a user attribute called "technics keyboards". Members choose from a long list of keyboards when they sign up. Let's say they tick the "KN5000" box.
Let's say once a day, the job runs. It is searching for the text "KN5000" in the users' information and if it finds that text it adds the user to a pre-made group called "KN5000".
I could then make a magic job for each type of keyboard, if I want to have a group for that keyboard.
I don't want to be very fussy about it, if the text "KN5000" (for example) is found associated anywhere in a user's information then I would add them to the KN5000 group because it would mean that they have an interest in that particular model.
Feasible?
Thanks,
Mike
I hope you're well.
Do you think it would be possible for be to use Magic Job to search through my users information and add them to a group depending on what text is found?
So, for instance here's what I'm thinking. I have a user attribute called "technics keyboards". Members choose from a long list of keyboards when they sign up. Let's say they tick the "KN5000" box.
Let's say once a day, the job runs. It is searching for the text "KN5000" in the users' information and if it finds that text it adds the user to a pre-made group called "KN5000".
I could then make a magic job for each type of keyboard, if I want to have a group for that keyboard.
I don't want to be very fussy about it, if the text "KN5000" (for example) is found associated anywhere in a user's information then I would add them to the KN5000 group because it would mean that they have an interest in that particular model.
Feasible?
Thanks,
Mike
| Type: | Discussion |
|---|---|
| Status: | Archived |
Thanks for the detailed response John. That sounds ideal to me. I'll buy those very soon... will need a few pointers from you please.
Many thanks,
Mike
Many thanks,
Mike
PS, yes a lot of signed up members. Over 2800 now.
You will definitely need the queueable job. 2800 would be too many for a single pass job, but well within the capability of a queueable job when sliced into groups of 10, or maybe a few more.
Hi John,
In testing, I've filtered all users who have a specific attribute containing a specific value as follows. Is there a way of adding each of them to a group? (I've obviously substituted names below). I'm using the '2000' value at the moment because I know there are less than 2000 users who meet the criteria.
I can only see a way to adding the current user to a group at the moment.
Many thanks,
Mike
In testing, I've filtered all users who have a specific attribute containing a specific value as follows. Is there a way of adding each of them to a group? (I've obviously substituted names below). I'm using the '2000' value at the moment because I know there are less than 2000 users who meet the criteria.
I can only see a way to adding the current user to a group at the moment.
LIST_ALL_USERS_WITH_FILTER_CONTAINING "an_attribute" "my_value" 2000 APPLY_EACH SET "temporary_group" AS_GROUP what_goes_here? END_APPLY_EACH
Many thanks,
Mike
The only symbol currently available to add a user to a group is ADD_CURRENT_USER_TO_GROUP
However, that symbol was deliberately limited to only work with the current user. Not any user.
Rather than a magic job, a way round it is to put an MD block with expression somewhere that is executed as a user browses the site, so users get added to the group next time they visit.
The other solution would be to write php for a new symbol that adds a specified user to the group.
However, that symbol was deliberately limited to only work with the current user. Not any user.
Rather than a magic job, a way round it is to put an MD block with expression somewhere that is executed as a user browses the site, so users get added to the group next time they visit.
The other solution would be to write php for a new symbol that adds a specified user to the group.
Just read back through the history of this. I will add the symbol to my to-do list and get an update of BMD out that can solve the problem for you.
John,
Thanks I really appreciate that. I think it would be beneficial to have the 'matching pair' to be able to remove a user from a group, just as you do with the current user (though I don't need it... yet).
I'm very gradually getting to grips with your 'Magic Language'! I'm in awe!
Thanks again,
Mike
Thanks I really appreciate that. I think it would be beneficial to have the 'matching pair' to be able to remove a user from a group, just as you do with the current user (though I don't need it... yet).
I'm very gradually getting to grips with your 'Magic Language'! I'm in awe!
Thanks again,
Mike
Good point, I will do the pair.
v1.7.1 of BMD provides the new symbols.
Hi John,
Fast work!
I'm using the following:
And I get this error for each iteration through the list of users:
error MagicDataSymbolsAddUserToGroup: current user is unknown
I'm getting a nice list of users and it is finding the correct group ID.
Is my expression correct?
Thanks,
Mike
Fast work!
I'm using the following:
LIST_ALL_USERS_WITH_FILTER_CONTAINING "technics_keyboards" "kn7000" 2000 APPLY_EACH SET "temp7" AS_GROUP ADD_USER_TO_GROUP END_APPLY_EACH
And I get this error for each iteration through the list of users:
error MagicDataSymbolsAddUserToGroup: current user is unknown
I'm getting a nice list of users and it is finding the correct group ID.
Is my expression correct?
Thanks,
Mike
You need the user before and group after (or vice versa).
Some tests from my dev system
Some tests from my dev system
SET JohntheFish AS_USER ADD_USER_TO_GROUP ( SET Administrators AS_GROUP )
SET Administrators AS_GROUP ADD_USER_TO_GROUP ( SET JohntheFish AS_USER )
In your expression, speed would be best using a memory to save working the group out inside the loop:
SET "temp7" AS_GROUP SAVE 'mgroup' LIST_ALL_USERS_WITH_FILTER_CONTAINING "technics_keyboards" "kn7000" 2000 APPLY_EACH ADD_USER_TO_GROUP ( SET 'mgroup' RECALL ) END_APPLY_EACH
That didn't work for some reason. I have the following working, adding and removing two users from the temporary group, is there any reason not to do it this way?
I've tried the above in a Magic Job... Will read the docs about Magic Queuable Job next!
Cheers,
Mike
LIST_ALL_USERS_WITH_FILTER_CONTAINING "technics_keyboards" "KN7000" 2 APPLY_EACH ADD_USER_TO_GROUP "temp7" END_APPLY_EACH
LIST_ALL_USERS_WITH_FILTER_CONTAINING "technics_keyboards" "KN7000" 2 APPLY_EACH REMOVE_USER_FROM_GROUP "temp7" END_APPLY_EACH
I've tried the above in a Magic Job... Will read the docs about Magic Queuable Job next!
Cheers,
Mike
I've edited that.
Once the users are in the groups (which I've been doing manually up to now) I then use Magic Data to show them relevant links (particular to the keyboards that they own) in their profiles and so on. They also get a group badge thanks to the Discussions Addon.. and much more planned :)
Once the users are in the groups (which I've been doing manually up to now) I then use Magic Data to show them relevant links (particular to the keyboards that they own) in their profiles and so on. They also get a group badge thanks to the Discussions Addon.. and much more planned :)
Such as... been a member for 1 year (2 years, 3 years...), add to a group, get a badge.
Programming things like that is well beyond me but I can cope with these sorts of things given a bit of thinking time thanks to Magic Data!
Programming things like that is well beyond me but I can cope with these sorts of things given a bit of thinking time thanks to Magic Data!
What you have is just as good, but will be a bit slower when you loop through 2000 users because it has to work out the group each time. If you start to run into CPU execution limits, every bit of speed will count.
The way I suggested with the group worked out first and put into a memory is theoretically faster, because the details of looking up the group in the database are only worked out once, not 2000 times.
The most likely reason for it not working would be missing white space, such as not having a space both sides of ( or ) .
The way I suggested with the group worked out first and put into a memory is theoretically faster, because the details of looking up the group in the database are only worked out once, not 2000 times.
The most likely reason for it not working would be missing white space, such as not having a space both sides of ( or ) .
John,
Tries to use the results from the previous line. So, it seems to pick up the group. Because the listed users aren't yet in the group it returns null. I think. The group id is 52.
Gives me "no users"
------
On a related subject, having returned a numerical list of user IDs, is there a way to sort them in numerical order? Because I could then REVERSE the list and limit it to just add the most recently signed up users (the others having been previously added).
I'm also seeking a way to list users who signed up before or after a particular date, or 1 year ago, 2 years ago etc.
I've read through all symbols (took a while!) and can't find anything to help with these. Maybe I could fund you to write symbols to assist, if you think they would be useful too?
Cheers,
Mike
LIST_ALL_USERS_WITH_FILTER_CONTAINING
Tries to use the results from the previous line. So, it seems to pick up the group. Because the listed users aren't yet in the group it returns null. I think. The group id is 52.
Evaluating symbol [LIST_ALL_USERS_WITH_FILTER_CONTAINING] using previous result [52] in context [Group]; Remaining: [technics_keyboards kn7000 2000 APPLY_EACH ADD_USER_TO_GROUP ( SET mgroup RECALL ) END_APPLY_EACH]
Gives me "no users"
------
On a related subject, having returned a numerical list of user IDs, is there a way to sort them in numerical order? Because I could then REVERSE the list and limit it to just add the most recently signed up users (the others having been previously added).
I'm also seeking a way to list users who signed up before or after a particular date, or 1 year ago, 2 years ago etc.
I've read through all symbols (took a while!) and can't find anything to help with these. Maybe I could fund you to write symbols to assist, if you think they would be useful too?
Cheers,
Mike
Good point about the LIST_ALL_USERS_WITH_FILTER_CONTAINING picking up the group.
You could try inserting a dummy subexpression to destroy that group/context. (eg CONTEXT ' ' NULL ), or something that is both useful and returns the context.
You can use
SORT_BY to sort a list by whatever is evaluated
FILTER_LIST to filter a list by whatever is evaluated returning a true-ish result
For sorting by user ID, I would experiment with:
for the most recent 100 by user ID. You may need to insert something into the sort to prevent it being optimised:
To remove those not already in a group
However, the ADD_USER_TO_GROUP symbol also contains that optimisation, so doing the above filter is likely less efficient than simply doing ADD_USER_TO_GROUP.
Good idea about a symbol for returning the registration/join date. However, evaluating and comparing dates is considerably more time-expensive than a simple numeric gate as per the sort/limit.
You could try inserting a dummy subexpression to destroy that group/context. (eg CONTEXT ' ' NULL ), or something that is both useful and returns the context.
You can use
SORT_BY to sort a list by whatever is evaluated
FILTER_LIST to filter a list by whatever is evaluated returning a true-ish result
For sorting by user ID, I would experiment with:
SORT_BY END_SORT_BY REVERSE LIMIT 100for the most recent 100 by user ID. You may need to insert something into the sort to prevent it being optimised:
SORT_BY AS_UID END_SORT_BY REVERSE LIMIT 100To remove those not already in a group
FILTER_LIST
IN_GROUP 'groupname' NOT
END_FILTER_LISTHowever, the ADD_USER_TO_GROUP symbol also contains that optimisation, so doing the above filter is likely less efficient than simply doing ADD_USER_TO_GROUP.
Good idea about a symbol for returning the registration/join date. However, evaluating and comparing dates is considerably more time-expensive than a simple numeric gate as per the sort/limit.
PS. Listing users would, I suspect, already return the list in database order (ascending ID), so the sort may be completely un-necessary and all you need to do is reverse. Its worth experimenting in the symbol tester to check.
Wow genius!
No, I think they are returning in 'activity' order that was my problem. I was experimenting with SORT_BY and didn't think to use AS_UID. Sorted!
Now, on the signup date issue, as I can search in the dashboard for signup dates (ordering users by signup date in the user search) I can also establish a user ID that relates to a specific date. Then using MD I can generate a list of user IDs, in numerical order, reverse it if necessary, truncate it according to the previously found user ID and add an attribute! Brilliant.
No, I think they are returning in 'activity' order that was my problem. I was experimenting with SORT_BY and didn't think to use AS_UID. Sorted!
Now, on the signup date issue, as I can search in the dashboard for signup dates (ordering users by signup date in the user search) I can also establish a user ID that relates to a specific date. Then using MD I can generate a list of user IDs, in numerical order, reverse it if necessary, truncate it according to the previously found user ID and add an attribute! Brilliant.
Just documenting this in case it helps someone.
I now have a series of pages in my personal section of my site. This is an area that I keep for such things, only available to the superadmin.
Each page has a Magic Data Direct block that will filter and add the most recently-joined users to the relevant group.
Once I'm 100% sure that all is working fine I plan to use Flexjob Scheduler and Cache Filler to process the pages once a day for the most recent ten users who satisfy the criteria.
I could alternatively render the pages with MD or use Magic Job but I like the above.
This will save me a few hours per month and means that users only have to wait around 24 hours for their profiles to be updated with lots of useful information that is directly relevant to them, instead of when I have the spare time. It also precludes potential mistakes when I'm feeling tired.
I now have a series of pages in my personal section of my site. This is an area that I keep for such things, only available to the superadmin.
Each page has a Magic Data Direct block that will filter and add the most recently-joined users to the relevant group.
Once I'm 100% sure that all is working fine I plan to use Flexjob Scheduler and Cache Filler to process the pages once a day for the most recent ten users who satisfy the criteria.
I could alternatively render the pages with MD or use Magic Job but I like the above.
This will save me a few hours per month and means that users only have to wait around 24 hours for their profiles to be updated with lots of useful information that is directly relevant to them, instead of when I have the spare time. It also precludes potential mistakes when I'm feeling tired.
Thanks for writing that up. I like your trick with pages, MD blocks and cache filler.
As you will have similar code across a range of groups, have you looked at using an MD snippet to abstract the common code?
You could set up your parameters in some memories, run the snippet (that would adapt itself to the parameters from memories), and get the result.
As you will have similar code across a range of groups, have you looked at using an MD snippet to abstract the common code?
You could set up your parameters in some memories, run the snippet (that would adapt itself to the parameters from memories), and get the result.
I'll definitely look into that John... I started the thread last October and it has taken me until now to learn enough about MD to get this far, due to time-constraints mostly, and I'm enjoying learning about it.
What might be good is something like "Magic Named Jobs", so that you could schedule multiple Magic Jobs. I've been keeping text files of Magic Jobs so that I can run them when I want. I can't write one long Magic Job as such because I might not want to run everything at the same time. Maybe I've missed something... or maybe that's a new addon!
What might be good is something like "Magic Named Jobs", so that you could schedule multiple Magic Jobs. I've been keeping text files of Magic Jobs so that I can run them when I want. I can't write one long Magic Job as such because I might not want to run everything at the same time. Maybe I've missed something... or maybe that's a new addon!
The core associates jobs with jobs classes 1:1, so it couldn't be separate jobs without separate classes or some significant surgery to the core.
However, you have sparked an idea. At the time I developed Magic Job, snippets didn't exist and that has just given me an idea. The job URL could have an optional snippet name as a parameter, then you could execute any snippet as a job. I will add that to my ideas list.
The disadvantage is that to add the parameters it would need to be run as an external cron job, it couldn't be run through the core jobs runner or flexjob.
However, you have sparked an idea. At the time I developed Magic Job, snippets didn't exist and that has just given me an idea. The job URL could have an optional snippet name as a parameter, then you could execute any snippet as a job. I will add that to my ideas list.
The disadvantage is that to add the parameters it would need to be run as an external cron job, it couldn't be run through the core jobs runner or flexjob.
Hi Mike
Just had another thought on this. Do you have Blocks by AJAX? It includes some symbols for interrogating post/get/request parameters.
So you could have a single Magic Job that implemented a dispatcher using MD to run a snippet specified in a job get string parameter.
For example (untested)
Then run the job via cron with a string like
For security, you may want to add a check of the allowed snippet names in the job.
Just had another thought on this. Do you have Blocks by AJAX? It includes some symbols for interrogating post/get/request parameters.
So you could have a single Magic Job that implemented a dispatcher using MD to run a snippet specified in a job get string parameter.
For example (untested)
SET "job_snippet" FROM_GET_DATA SAVE 'dispatcher snippet name' APPLY_SNIPPET ( SET 'dispatcher snippet name' RECALL )
Then run the job via cron with a string like
http://www.yourdomain.co.uk/index.php/tools/required/jobs/run_single?auth=e9856714958afeb&jID=X&job_snippet=any_snippet_name
For security, you may want to add a check of the allowed snippet names in the job.
SET "job_snippet" FROM_GET_DATA SAVE 'dispatcher snippet name' SET "allowed_snippet_name1,allowed_snippet_name2,allowed_snippet_name3" AS_LIST LIST_CONTAINS ( SET 'dispatcher snippet name' RECALL ) ZERO_AS_THEN_END 'Snippet not authorised' APPLY_SNIPPET ( SET 'dispatcher snippet name' RECALL )
Hi John,
I have blocks by ajax but I don't know if I have cron. I'll find that out and hopefully be able to try it, will let you know.
I'm delighted right now because I have the main keyboard groups set up, the members have their badges, their profiles link to specific pages for the keyboards they own, their countries are autodetected and I even have their flags and countries next to their avatars in the core discussion forums addon. All going great!
Cheers,
Mike
I have blocks by ajax but I don't know if I have cron. I'll find that out and hopefully be able to try it, will let you know.
I'm delighted right now because I have the main keyboard groups set up, the members have their badges, their profiles link to specific pages for the keyboards they own, their countries are autodetected and I even have their flags and countries next to their avatars in the core discussion forums addon. All going great!
Cheers,
Mike
I've discovered another point of note. Because the filtered user list needs to be sorted before being reversed, it needs to be complete. So it can't be limited much until the next step. Therefore I'm currently limiting the filtered user list (for each attribute, or keyboard model) to 1000 (more than large enough for now) and then adding the most recent 100 users to the group. I'll reduce this to 10 or less in the near future in order to reduce database interaction.
v1.12 of Uber List provides:
- AND_EXCLUDE - exclusion filtering attached to lists
- AND_SORT_BY - sorting at the database level attached to lists
- AND_EXCLUDE - exclusion filtering attached to lists
- AND_SORT_BY - sorting at the database level attached to lists
v1.3 of Magic Job provides two new jobs:
Magic Snippet Job - specify a snippet to execute in a job parameter.
Magic Flush and Fill Job - flush the cache and pre-fill pages according to a Magic Data list. A slicker MD powered combination of the cache vac and cache fill jobs.
Magic Snippet Job - specify a snippet to execute in a job parameter.
Magic Flush and Fill Job - flush the cache and pre-fill pages according to a Magic Data list. A slicker MD powered combination of the cache vac and cache fill jobs.
That is just the sort of problem Magic Job is good for solving. To add
users to a group, you would need to also have Black Magic Data - that is
the extension that contains all the potentially dangerous stuff that can
write to the database.
Rather than a separate job for each keyboard (which is possible by
copying the job a few times), it would be more efficient to check a
specific user for all keyboards/groups at the same time.
The difficult part will be that you probably have thousands of users and
a job processing all of them in one go would time out, whether its one
keyboard or 100 you are checking.
To cope with that there is also the Magic Queue-able Job, where all the
users get listed at the start, but then one user is processed with each
slice of the queue.
The process would be roughly:
List all users and start the queue
For each user:
List all possible keyboard groups.
For each group name, see if the user has that keyboard/group name in
their text
Add user to group
By default I think the number of queued items (users) processed is 10
for each queue slice, but you may need to adjust that lower or higher.
John
On 02/10/2015 22:32, concrete5 Community wrote: