concrete5 5.7 Developer Snapshot Alpha '14 (Major Updates)
Permalink 2 users found helpfulhttp://github.com/concrete5/concrete5-5.7.0/...
We have set up this version in its own repository! We're going to be moving all 5.7 development into this repository.
Obviously, since this is still just in Github, this is flexible – not a "release," per se. If you've been following 5.7's development you have some idea what to expect. We've made some progress on skinning the dashboard, reducing some of the outright broken functionality, tightening up some of the edit interface and the new panels, and adding functionality back into the file manager and sitemap. It's very far from being stable and production ready, but it's ready to play with. All other out-of-date, inaccurate 5.7 branches have been removed from the concrete5 repository; at this point, it's just 5.7.0-wip and master, which should make it much clearer which repository to work on.
If you just want to give the new UI a spin, feel free to head on over to Github and download a zip. However, if you've ever written custom concrete5 code, whether for a site or for an add-on, stick around. Just a glance at the repository will make it obvious that quite a bit has changed. I kept it private because honestly I wasn't sure I was going to be able to pull it off.
As many PHP developers probably know, a lot of nice things have been happening in the PHP world over the past couple years. PHP 5.3 (and beyond) have brought some great advances to the language; package managers like Composer have helped normalize the re-use of code; organizations like the PHP Framework Interoperability Group have grown, comprised of members from many different open source projects, in an effort to agree on certain framework standards. It's never been a better time to be a PHP developer.
concrete5 has always prided itself on having smart, well-architected code (if not always the most consistently documented). Unfortunately, we haven't been able to take advantage of these advancements, as we've made the conscious choice to stick with PHP 5.2. However, with the impending release of concrete5 5.7, we made the choice to begin requiring PHP 5.3. For months, all that meant was that new code could take advantage of 5.3 features like late static binding. We had no plans to refactor or rework old code.
However, as 5.7's release date began to slip, and it became clear that that we weren't going to be as easily backward compatible as in the past, it became obvious what we had to do: if we did a release that wasn't backward compatible, it was going to be exceedingly difficult to do another release later that took advantage of these new features, and make [b]that[/b] release also not backward compatible. That wasn't going to work. We had to move it all into 5.7; if you have to pull the bandaid off, do it all at once.
With that in mind, here are some of the substantive changes for developers in 5.7:
## New in 5.7 – Prior to April 2014.
Here are the following new developer features in 5.7 that won't be news to anyone who has checked out its code throughout its development:
* Completely new routing system, based on the Symfony2 Routing Component. This will be used to power all new 5.7 interfaces. concrete5 internal dialogs and panels are now using this new routing and MVC component. Tools files are officially deprecated (although there are still lots of them.) Ultimate Goal? Move all UI and processing into views and controllers, and remove tools.
* Completely new Request system, based on the Symfony2 Request component.
* Completely rearchitected concrete5 MVC system, and the removal of lots of weird no longer necessary hacks (on_page_view() methods in blocks, etc...)
* New Assets system, which lets you group assets and require asset groups. Useful for core and add-on developers to ensure that assets are included when they ought to be, and not duplicated.
* New Response library for edit requests, based on the Symfony2 Response library. Much easier JSON-based development for core and add-on developers.
* New Cookie library based on the Symfony2 Cookie component.
* Much improved JavaScript throughout the code, including the use of underscore templating and better coding practices. Much old concrete5 JavaScript code has been upgraded and reworked.
These upgrades were going to be mildly annoying to those who have code in existing concrete5 sites. For example, the new MVC system requires that all page controllers extend the PageController, rather than just the Controller class.
## New in 5.7 - April 2014
Here's where we rip the bandaid off:
* Completely reorganized code. No more arbitrary "models" and "libraries" and "helpers" directory. Instead there is one "core" directory.
* Completely namespaced code throughout concrete5 entirely, including the core directory and blocks, attributes, packages, etc...
* No more empty classes in the concrete/ directory. No more empty block controllers that extend other classes.
* Removed Loader::model and Loader::helper. Hell, remove the need for Loader entirely. Autoloading is standard based on a concrete5 modified PSR-4 autoloading (from the PHP-FIG Group)
* Completely rearchitected dispatcher, including the cleanup of years of configuration code.
* The beginnings – the barest, barest beginnings – of test driven development surrounding some of the new code (although I don't think it currently passes – and I know the old tests don't.)
* Dispatcher extends the Laravel IoC Container, allowing for sensible binding of services (which replace helpers) for lazy loading and overriding. Easily override core classes with code like this:
Core::bind('helper/feed', function() { return new \Application\Core\CustomFeedHelper(); });
* Alas – and it does kind of make me sad to say it – there is no more ADODB in concrete5. Anywhere. We have integrated Doctrine DBAL and Doctrine ORM. We have the BlockType class running through Doctrine ORM as a proof of concept, but mostly we're still doing direct database access. I'd love to see what people can do with this. We have kept backward compatibility by writing a compatibility layer for Doctrine that extends their core classes. We also have a custom AXMLS parser for old blocks and packages. That actually didn't take that long to write.
* No more site_events.php, site_autoload.php, site_post.php. Instead, there's one "app.php" in your config directory that can contain anything. Put your class overrides, your theme path overrides, custom event definitions in it.
* The Events class extends the Symfony2 Event Dispatcher library, which is a much more structured and full-featured event framework. Events can be stopped from propagating, and work with custom event objects.
* A new Session class replaces direct access to the $_SESSION array, which should make it much more testable, and make database-backed sessions much easier to get working (in fact, I'd like to see someone do just that.)
* We have moved all overrides into an application/ directory. This looks cleaner and I'm sure smart developers will find a way to do something with this that I haven't anticipated yet.
* Certain static facade classes are available all throughout your sites, like Request, Session, Core (which creates classes) and more – you don't have to instantiate them with calls like Request::getInstance().
* All third party libraries are being fetched and kept in sync with Composer. This means many of them have been updated without even knowing it.
There's probably more to it than even that, but my fingers are tired and I think we're definitely into "TL;DR" material.
## What's Next?
1. We are going to continue developing 5.7 and 5.6.3, and release 5.7 when it's ready.
2. We would love your help cleaning up all the issues surrounding 5.7. There are still a few dashboard pages that don't work at all, and a fair number that aren't adhering to the new style guide. There are lots of other spots in the core that are also probably broken due to incomplete namespace integration.
3. We will offer a solution for engaging more with the community in getting this help. Whether that is Github Issues or an alternate issue tracker I don't know. It's not going to be concrete5.org's bug tracker yet – it's not quite ready for that. I don't think Github's issues are up to snuff so I'm looking forward to turning them off once we have a place to direct people to.
4. I will post a series of deeper looks into some of this functionality on my personal website, with the idea that these articles will eventually make it into concrete5.org when our new 5.7-centric site is unveiled.
5. I will also post a migration guide for a very simple package from the marketplace. I downloaded a free add-on that had a block and a couple dashboard pages, and figured out what was required to get it working.
Please – give us your thoughts. It's exciting to have the underlying foundation of concrete5 looking as pretty as the exterior.
Although I could, right now, I just don't want to delete stuff, install Composer, turn on short tags, etc, etc.
How far away are we from getting something that will have the normal install and be able to start working on blocks and add ons? Seems like this right now is more for the core hackers than the developers.
I uploaded this new Git version to our server. And I installed composer as described with no errors. But no luck!
I get the following error both in Firefox and Chrome when I start the website:
Firefox has detected that the site redirects the request to itself in a way that prevents it from ever being finished.
# Server Software: Apache/2.2.22 (Debian)
# Server API: fpm-fcgi
# PHP Version: 5.4.4-14+deb7u8
# MySQL Version: 5.5.35-0+wheezy1
Away for no reason and never came back.
This develop server is a exact replica of our live servers, and I have never seen any isues with it before. It is running about 30 test domains.
Any idea would be appreciated.
On my machine, composer install failed. Removed all empty folders in concrete/vendor directory, then it worked.
2) What's Next#3
Modernized concrete5 seems so great and I'm excited, but most of problems posted on "Welcome to the future of concrete5" thread still remains. Should I wait for coming new solution to post issues about 570-wip?
About composer: What's your plan with packages? Still use the custom installer for package and keep them in "packages"? I guess it would be more composer like to put them in "vendor", which would also make it possible to get rid of the custom installer. The term "vendor" is imho a bit strange for packages though..
Plus the naming collisions with concrete5's composer in the docs would be brutal ;-)
http://www.youtube.com/watch?v=9qpE8bW1iAA...
I cannot install the current WIP branch, though. I get a lot of SQL exceptions from the PDO handler, e.g. trying to save booleans to INT-type column and "colum 'timestamp' does not have a default value".
Are you accepting / will you be accepting pull requests currently to the WIP branch or should we wait for you to come up how to manage the community workflow? E.g. I would know how to fix the exceptions I'm having during installation. There are some backwards compatibility things missing also from the XML parser, the "no default value" is one of these.
About the Remo's point about packages, I would keep the "package" concept in concrete5 as is but I would add possibility to include composer packages into concrete5 packages. I'm not sure if this should be centralized somehow... e.g. if 2 packages require the same version of the same composer package, it wouldn't need to be (down)loaded twice.
Code-wise, it looks overall really GOOD! Can't wait to dig in more.
FYI: We're investigating moving this to its own repository with its own issues system (which we will open up) – so sit tight and we'll probably have something to say about this this morning. In the meantime, if you figure out what some of the issues are PM me and I'll check in fixes.
One thing to check: up your RAM allotments. Possibly by a lot. I don't know why, but the custom Doctrine DB XML parsing code that we have working takes FOREVER to install all the block types, and I'm wondering if something dumb in there is increasing resource usage. It's also possible that Doctrine itself is just much, much heavier than ADODB ever was (sigh.)
Sent from my iPhone
> On Apr 22, 2014, at 7:36 AM, concrete5 Community <[email protected]> wrote:
PHP Version 5.3.25
Apache/2.2.22
MariaDB 5.5.5
[Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP Fatal error: Maximum execution time of 300 seconds exceeded in C:\\Work\\concrete5-5.7.0\\web\\concrete\\vendor\\doctrine\\dbal\\lib\\Doctrine\\DBAL\\Driver\\PDOStatement.php on line 33, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP Stack trace:, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 1. {main}() C:\\Work\\concrete5-5.7.0\\web\\index.php:0, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 2. require() C:\\Work\\concrete5-5.7.0\\web\\index.php:2, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 3. require() C:\\Work\\concrete5-5.7.0\\web\\concrete\\dispatcher.php:33, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 4. Concrete\\Core\\Application\\Application->dispatch() C:\\Work\\concrete5-5.7.0\\web\\concrete\\bootstrap\\start.php:133, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 5. Concrete\\Core\\Routing\\Router->execute() C:\\Work\\concrete5-5.7.0\\web\\concrete\\core\\Application\\Application.php:289, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 6. Concrete\\Core\\Routing\\ControllerRouteCallback->execute() C:\\Work\\concrete5-5.7.0\\web\\concrete\\core\\Routing\\Router.php:58, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 7. Concrete\\Core\\Controller\\AbstractController->runAction() C:\\Work\\concrete5-5.7.0\\web\\concrete\\core\\Routing\\ControllerRouteCallback.php:18, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 8. call_user_func_array() C:\\Work\\concrete5-5.7.0\\web\\concrete\\core\\Controller\\AbstractController.php:94, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 9. Concrete\\Controller\\Install->run_routine() C:\\Work\\concrete5-5.7.0\\web\\concrete\\core\\Controller\\AbstractController.php:94, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 10. call_user_func() C:\\Work\\concrete5-5.7.0\\web\\concrete\\controllers\\Install.php:185, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 11. Concrete\\Core\\Package\\StartingPointPackage->add_users() C:\\Work\\concrete5-5.7.0\\web\\concrete\\controllers\\Install.php:185, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 12. Concrete\\Core\\Authentication\\AuthenticationType::add() C:\\Work\\concrete5-5.7.0\\web\\concrete\\core\\Package\\StartingPointPackage.php:182, referer: http://testing57.dev:8000/index.php/install [Tue Apr 22 15:23:45 2014] [error] [client 127.0.0.1] PHP 13. Concrete\\Core\\Package\\Package::installDB() C:\\Work\\concrete5-5.7.0\\web\\concrete\\core\\Authentication\\AuthenticationType.php:222, referer: http://testing57.dev:8000/index.php/install
I didn't have a closer look though.. Might be easy to fix.
http://github.com/concrete5/concrete5-5.7.0/...
It installs on a linux vagrant instance for me now, in addition to my OS X box. Barring that we'll try and figure out what's going on with it on Windows as we continue development.
I was able to git the dev and set up locally just fine.
Thanks for your hard work.
C
OS: Windows 7
Server: Apache/2.2.22 (Win32)
PHP Version: 5.3.14
DB: MySQL 5.1.50-community
I don't experience any timeouting but a lot of DB exceptions from the PDO handler. These all come during the installation.
Here's some of the exceptions:
#1
An exception occurred while executing 'insert into GatheringItemTemplates (gatTypeID, gatHandle, gatName, gatFixedSlotWidth, gatFixedSlotHeight, gatHasCustomClass, gatForceDefault, pkgID) values (?, ?, ?, ?, ?, ?, ?, ?)' with params ["1", "featured", "Featured Item", "6", "2", false, true, 0]: SQLSTATE[HY000]: General error: 1366 Incorrect integer value: '' for column 'gatHasCustomClass' at row 1
Fix:
/web/concrete/core/Gathering/Item/Template/Template.php:210
=> Add intval() around the boolean values
#2
An exception occurred while executing 'insert into CollectionVersionBlocks (cID, cvID, bID, arHandle, cbDisplayOrder, isOriginal, cbIncludeAll) values (?, ?, ?, ?, ?, ?, ?)' with params ["113", "1", "1", "Header", 0, 1, false]: SQLSTATE[HY000]: General error: 1366 Incorrect integer value: '' for column 'cbIncludeAll' at row 1
Fix:
/web/concrete/core/Page/Collection/Collection.php:738
=> Add intval() around the boolean value
#3
An exception occurred while executing 'INSERT INTO SystemImageEditorControlSets (scsHandle,scsDisplayName,pkgID) VALUES (?,?,?)' with params [{"0":"position"}, {"0":"Position"}, false]: SQLSTATE[HY000]: General error: 1366 Incorrect integer value: '' for column 'pkgID' at row 1
Fix:
/web/concrete/core/ImageEditor/ControlSet.php:112-114
=> Add intval() around the (possible) boolean value
#4
An exception occurred while executing 'INSERT INTO SystemImageEditorComponents (scsHandle,scsDisplayName,pkgID) VALUES (?,?,?)' with params [{"0":"text"}, {"0":"Text"}, false]: SQLSTATE[HY000]: General error: 1366 Incorrect integer value: '' for column 'pkgID' at row 1
Fix:
/web/concrete/core/ImageEditor/Component.php:112-114
=> Add intval() around the (possible) boolean value
#5
An exception occurred while executing 'INSERT INTO SystemImageEditorFilters (scsHandle,scsDisplayName,pkgID) VALUES (?,?,?)' with params [{"0":"grayscale"}, {"0":"Grayscale"}, false]: SQLSTATE[HY000]: General error: 1366 Incorrect integer value: '' for column 'pkgID' at row 1
Fix:
/web/concrete/core/ImageEditor/Filter.php:111-114
=> Add intval() around the (possible) boolean value
#6
Fatal error: Class 'Concrete\Core\File\Service\Imagick' not found in C:\dev\php\concrete5-5.7.0\web\concrete\core\File\Service\Image.php on line 201
Fix:
/web/concrete/core/File/Service/Image.php:201,211
// 201 $image = new Imagick(); // => $image = new \Imagick();
// 211 if($image->getCompression() == imagick::COMPRESSION_JPEG) // => if($image->getCompression() == \Imagick::COMPRESSION_JPEG)
#7
An exception occurred while executing 'update PageTypeComposerFormLayoutSetControls set ptComposerFormLayoutSetControlRequired = ? where ptComposerFormLayoutSetControlID = ?' with params [false, "2"]: SQLSTATE[HY000]: General error: 1366 Incorrect integer value: '' for column 'ptComposerFormLayoutSetControlRequired' at row 1
Fix:
Make the XML parser respect the default date/timestamp definitions from the AXMLS.
It went pretty far with these fixes but the last one is too big to fix at this moment for me.
I think if you use the Doctrine/PDO statement instead of "executeQuery()", you wouldn't get these "incorrect integer value" errors. I'm not sure though, but I would think so (from my previous experience).
And yeah, I'm at a bit of a loss as to how to do the timestamp field in Doctrine. Do you have any experience in that? I couldn't find any way to do CURRENT_TIMESTAMP in doctrine. I think right now we're just kind of kicking it down the road but obviously we have to fix it prior to release.
I'm also curious as to why these integer fixes that require intval() are happening on your environment and not mine. I don't doubt it but I'm wondering what's different in the abstraction layers between the two databases. Should we be defining them differently? Are we screwing up the queries somehow?
It would help a bit to hear more about your schedule plans on 5.7. I'm personally busy for the upcoming week or two, so I wouldn't have any time before that.
And yes, I've implemented the timestamp values in my own AXMLS => doctrine parser, so I can look back there to find out how to do it (it's the package I sent you earlier).
UPDATE: Afaik, I works on Windows
A bit of background: a while back I did an experiment to integrate docrtine into 5.6, so this is the result of that. It also has backwards compatibility functionality for the ADODB API and I actually managed to install concrete5.6 successfully on SQLite with that.
http://demo.mainiotech.fi/c5od.zip...
I can assure I'm not trying to install trojans on your machine(s). ;)
Working in concrete5
Check this:
/libraries/database/adodb/schema.php
And search for "$defaultCurrentTimestamp".
I've been using doctrine for 2 years.
https://dev.mysql.com/doc/refman/5.1/en/sql-mode.html#sqlmode_strict...
Fix:
http://stackoverflow.com/questions/8874647/general-error-1366-incor...
I got it installed with that settings change in MySQL, i.e. disabling STRICT_TRANS_TABLES.
But I still think it should support that setup as well, I think many might have it set.
Thanks so much for the timestamp and data dictionary code – I don't know how long it would've taken me to figure that out. There are probably some other edge cases for the AXMLS -> Doctrine legacy parsing that we haven't addressed in our parser, simply because the core doesn't use those types.
I can check in those edge cases later on when I have time. Actually the parser I've written was 100% for the 5.6 core, so everything you see there at least WAS used in the 5.6 core (and I wouldn't be surprised if some of it was used by add-ons as well). I'll need to take a closer look at your parser and see what's missing there.
I'd also like to put in some effort to make concrete5 database agnostic, at least for SQLite (that is already mostly done in my package for 5.6) and also PostgreSQL (it requires some more work on the database item list searching functionality, I've not yet done this).
obviously everything is so different.
Would you be willing to revamp your FAQ app with one or two attributes and a page_type added and maybe a block so that we can work towards converting our apps?
ChadStrat
Just starting with basic stripdown. nothing in my app except the db.xml.
That seems to fail quite horridly.
Do you have any was->is resource for like db schema's for packages? Or can we get something in that vein?
C
There's a difference between a DB entity and a model and this should be considered in the core as well:
http://symfony.com/doc/current/book/doctrine.html#creating-an-entit...
I think this defines it the best:
Entities represent information, Models contain business logic
Source:http://blogsh.de/2011/09/08/models-vs-entities/...
Not saying it's the most useful or complete example out there, but it was an easy place to start, given the way that BlockTypes used to work.
Another point, I think "getBlockTypeID()" should lie in the entity itself, it's not business logic. Although, I'm not seeing the "big picture" very clearly currently, I've not yet wrapped my head around the new core.
C
Prioritize them.
What's the proper way to go about implementing a json ajax request within a package block now that legacy tools files are deprecated (and don't seem to work from a non-core block anyway).
-Steve
Or am I missing something?
-Steve
I had an issue installing this after downloading it from github. That's because our third party libraries are now going in as git submodules (which was not intended.) So here's what you should do.
1. Re-clone you concrete5 repository. Make sure there's no concrete/vendor/ directory.
2. cd web/concrete/ and run "composer install" (more info at getcomposer.org if you need to install composer.)
3. That should install the various dependencies necessary.
And, as always, make sure you have PHP short tags enabled. Hope this helps everyone! I've updated the README in the 5.7.0-wip repository to make this clearer.