So when I say I work with Umbraco, people kind of always go, "ahh Umbongo Umbongo, they drink it in the Congo" referencing the TV advert jingle for sugar and fruit flavoured drink 'Umbongo' from the mid-nineteen eighties; which I can't deny is a particulary catchy jingle, but one I've always felt slightly self-concious singing along to. On the other hand, the Kia-Ora jingle of the same vintage and ouvre is as catchy as a rash, and I'm much more comfortable with it in the sing-a-long stakes. So when I had to name my blog which would largely concern Umbraco subject matter and I HAD to pick an eighties fruit based tv advert to reference, I chose the Kia-Ora one. Now I could have gone with the ad strapline 'We all adore a Umbraco' but somehow 'Umbraco, too orangey for crows' made more sense (nonsense). So sorry for wasting your time with this explaination, but I think it gives a fairly good indication about whether it's worth reading any of the blog posts...
Written by @marcemarc on Wednesday, 31 January 2018
So we've inherited a new client, they have two sites, it is a travel company, and as is so often the case, one site is an obvious clone of the first site, and you know somebody has just said we just want that again but with a different logo... and they are so close... twins almost, and although people still talk about the 'main' site... like it was a favourite child - well you still wouldn't expect them to have any major differences...
Redirects are important for this company. SEO is everything. the 301 redirect is king (or queen).
"we want to be able to have the same 'redirects thing' on this one, as we have on the main site"
"Sure.. no problem", I figure I just install whatever package is used on the 'main site' on the twin site, it all should just work...
".. and actually the tracking thing's not setup properly on the main site, it works on the second site though, you know automatically creating a redirect if we rename a resort.."
"ahh...", alarm bells begin to ring...
So the 'main site' has Simple 301 redirect package installed the 'second site' doesn't.
...On the one hand, I can see how the lack of symmetry might really annoy some people...
So the 'second site' without the Simple 301 package relies on the Core Redirect Url Management dashboard for it's 301 Redirecting powers, and the editors have fathomed the 'rename-rename-remove' method from 'part 1' to create redirects.
(They have no connection though with the other company, how do editors pass these secrets across organisations?... it's like whale song or something).
If I install, as requested, the Simple 301 package on the 'second site' it will turn off the Redirect Url Management tracking functionality and any redirects already created with that mechanism will fail...
... I can make the core Redirect Url Management tracking happen on the 'main site' but only by uninstalling the Simple 301 package, and losing all the redirects that have been already created with that package...
... what? ...
I said it was a RUM do... Part Two...
Why don't existing 301 and Tracking packages play nicely with the RUM dashboard?
When we squeezed the Redirect Url Management tracking functionality and barebones dashboard into the release of 7.5 at that retreat, I mentioned in Part 1 - we had a horrible, 'end of day' thought... What if?
What if somebody already has installed one of the many existing 301 redirect management packages, and they upgrade to Umbraco 7.5 and their site suddenly has two redirect mechanisms... what will happen?
Well, if one contains a redirect to a nice url of /about-us from a not-so-nice url /aboutus and the other mechanism contains a redirect from the not-so-nice url /aboutus back to the nicer url /about-us ...well...
FIGHT!
...a redirect loop will create a continuous game of redirect tennis™ between the two mechanisms until well, I guess, well the server melts... creates a mini black hole, unicorns die etc... the internet is broken.
How, How can this be?
It all depends on the incoming request pipeline of Umbraco and where the different redirect mechanisms choose to look up and try to perform the redirect.
The Umbraco request pipeline is a series of IContentFinders... individual rules to find Umbraco content based upon the incoming url ... executed one after another in a queue, until content is found.
...and what we didn't know, at the very last minute, is quite where in the request pipeline each individual plugin/package/redirect mechanism was plumbed, in relation to our new ContentFinderByRedirectUrl implementation...
so very pragmatically this code was added...
// if any of these dlls are loaded we don't want to run our finder var dlls = new[] { "InfoCaster.Umbraco.UrlTracker", "SEOChecker", "Simple301", "Terabyte.Umbraco.Modules.PermanentRedirect", "CMUmbracoTools", "PWUrlRedirect" };
// assuming all assemblies have been loaded already // check if any of them matches one of the above dlls var found = AppDomain.CurrentDomain.GetAssemblies() .Select(x => x.FullName.Split(',')[0]) .Any(x => dlls.Contains(x)); if (found) ContentFinderResolver.Current.RemoveType<ContentFinderByRedirectUrl>(); }
Essentially if you have one of the existing redirect plugins/package dlls in your site, the core Redirect Url Management dashboard, does not want to play, it is turned off, it goes home in a sulk... and removes itself from the content finding pipeline...
... on the upside, nobody's servers melted after upgrading to 7.5! - but quite a few people were puzzled about this new functionality that plainly did not work... which was a shame, especially for those whose motivation for upgrading to 7.5 was the thought of 'ditching their reliance on a 3rd party tracking package and taking advantage of this new core thing' ...
Dilemma
So now you understand my dilemma for the twin travel sites? (two sites remember! - not a single site dedicated to travel destinations for twins)...
... and why installing simple 301 or any other package to service their SEO fueled 301 redirect cravings will break the core tracking functionality and vice versa...
So I figured I needed to do some investigation... just where in the request pipeline does each of these packages kick in?...was there really a problem or just a last minute 'what if'?
Why don't you just use UrlTracker mate?, it does tracking AND redirects...
If you do have UrlTracker installed, please make sure you either have 404 tracking turned off, or ignore common 404 requests in the app settings, or have a maintenance plan to remove entries if you are not going to redirect them... or... this will happen:
basically, the 404's are tracked in the same db table as the 301s, each tracked 404 is a new row, which if you are not careful, build up over time, and cripple the speed of the dashboard. (UrlTracker appears not to be actively maintained but secretly it is! - You can find it here: https://www.nuget.org/packages/UrlTracker/ there are later versions on Nuget than on Our.Umbraco)
It has the ability to 'force redirects' which then occur before the Umbraco pipeline has its turn, and 'un-forced' redirects that occur after the Umbraco pipeline has executed, and it has regex pattern matching too - but it wouldn't run nicely alongside the Redirect Url Management dashboard... we'd have Url changes tracked in two locations. and we could have the potential 'redirect loop meltdown problem', we're so worried about... it doesn't currently use an IContentFinder to hook into the Umbraco request pipeline but runs instead using a HttpModule on every request...
... if you have UrlTracker installed then it makes sense to turn off the core Redirect Url Management functionality but...
... but that's part of the core now, and the nub of this is people want to use the 'core thing', regardless of whether it fulfills their needs, simply because it is in the 'core' ... There is encouragement that this will always be maintained, work with Umbraco deploy, Umbraco Cloud etc, and there is the expectation it will evolve... but yes, stop reading, just use UrlTracker, it isn't my point not to!
What about Simple 301?
It's simple - it doesn't try to do any automatic tracking of changes... and it's built in angularJS, which makes its UI fit with the paradigms of Umbraco 7 neatly, although the colours are wrong... also it's using an IContentFinder to register itself in the pipeline, looking good! ... but oh wait... the IContentFinder is registered at the beginning of the pipeline, so on every content request, redirects are being looked for, albeit the database request is being cached...but..
... I don't understand why this IContentFinder is being registered first?
for me you shouldn't be using a redirect system to override an existing Umbraco Url?, the redirects should occur after attempts to find genuine published content have been made... and having this IContentFinder for Simple 301 execute first and importantly before the RedirectUrlManagement IContentFinder is unfortunately just the set of circumstances that would enable a set of redirect tennis™ to be played between the two...
...so I tried it...
Not quite the end of the internet, no black holes, unicorns still exist, but pretty annoying for anyone visiting the URL and having the 301 loop cached in their browser... so I guess this is a no go too, am I going to have to write my own? [Insert link to part 3]
...and then I thought, but hang on, I've got access to the source of this, this is opensource...
... so as an experiment I've forked Simple 301, and changed where the IContentFinder is registered, and now it's plumbed in after the ContentFinderByRedirectUrl finder...
... now the redirect loop meltdown cannot occur...
Why?
if we think about the /aboutus -> /about-us -> /aboutus example - the Simple 301 can still redirect /aboutus to /about-us, and there can still be a redirect in the Redirect Url Management dashboard from /about-us but this can ONLY be to a published content item, and therefore the content finding content finders ahead of the redirect finders in the content request pipeline will find that published item first, no loop can occur, if the item being redirected to is subsequently un-published, the redirect is removed from the Redirect Url Management dashboard, so again no loop!
But why isn't the core tracking turned off by the presence of the dll?
... umm, well I changed the dll name in the fork...
So the world of 'pragmatism' cheers and drowns out the orbiting satellites of 'best practice' and 'common sense' and I have the RUM edition of Simple 301 running on the twin travel sites, editors can add custom SEO driven 301 redirects to their hearts content, and any re-names of published content are automatically tracked and 301 redirected to the new name by the core Redirect Url Management,
everything is more symmetrical... the twins are a little more identical...
... am I meant to say 'for the win' here?, I've seen other people say that kind of thing, so try to imagine I have seamlessly done so too.
So Part 2, not what you expected? it is hard in a trilogy, for the middle episode, to keep the plot moving of the overarching story arc and yet still manage to work as a standalone ep.
What I'm trying to say here in all this is: should this kind of redirection always be outside of the core? and if so, can we find a way to adopt one of the packages to work in harmony with the core tracking mechanism, we could then extend and improve the package if necessary, and just avoid the confusion of it not being clear why everything stopped working on one thing, when we installed another thing, because of its name?
Do we even need the additional level of redirection? would the additions in Part 1, being able to add to the Redirect Url Management dashboard manually, cope with most redirection scenarios that editors need to manage via the CMS?
Where should our 'sorting out redirection in Umbraco' development energies lie? Core or Package or...
...is there a third way? There is a Part 3...[add link to part 3]
NB: thanks to Wade Kallhoff for creating the simple 301 package, and apologies for experimenting with it like this.
Written by @marcemarc on Saturday, 13 January 2018
So silly things, although it’s easy to blame…
… And I’m going stupid once again, but I left it up to you.
Could these obscure ‘indie dance’ lyrics from the Paris Angel’s UK top 55 ‘hit’ in 1990 have been written specifically about the current situation for redirects in Umbraco?
No of course not, but it’s getting harder and harder for me to shoehorn a musical reference into my blog posts, and pick an appropriate youtube video to suit the subject matter or at least a bad pun of the blog post title, that well now, hmm, now, I’ve started picking the song first, and then I try to weave the blog post subject nonsense around the tune…
Let’s see how I get on, in theory there will be a series of posts about redirects here, reflecting the redirect related events in my life over the last three months, it seems like I nearly have enough for e-book.
But it is my New Year's Resolution to be more direct, and stop these whimsical digressions and observations, that make people forget what the original blog post was intended to be about by the time they reach the end of the article.
Let’s see how I get on.
Redirection Url Management
You may have noticed the introduction of the Redirection Url Management dashboard in Umbraco 7.5.x - (I accidentally had a hand in it, it sort of only exists (or at least the dashboard appeared in this version) because I was somehow invited to the Umbraco HQ retreat, and as a way of hiding during the 2nd day, I suggesting I’d be working on it, as we went around the group in the morning standup, as it sounded like a thing, even though it wasn’t a thing. http://issues.umbraco.org/issue/U4-8802 and it’s fun to see how very little of what I built made it into the actual dashboard… https://github.com/umbraco/Umbraco-CMS/commit/b4d5140d077c567e87423725e45cb0282d1c0b43#diff-1d1ef1a751b655e89ec108605bae6d65
but apparently the fact I had started it and work was in progress, was why it was polished up to squeeze into the release, every little helps, but maybe its genesis explains a little why the next question in your head is probably.
Why can’t I manually add redirects ?
The whole addition of the tracking was built as a minimum viable product in 48hrs, what was the smallest thing that could be built, and still make an improvement, for the single issue when the CMS knows a published url, has been changed, to issue a 301 redirect to the new url. The dashboard wasn’t part of that, the 301 redirects that get setup by this tracking of changes were going to be invisible… I figured that if Umbraco was going to start adding these redirects automagically, making them visible somehow to editors, would at least let people know what was going on…but mainly I’d have something to demo at the end of day two.
Adding redirects for a mechanism that only tracks changes doesn’t makes sense, especially when the day is ending and people had already started to mix rum cocktails - That doesn’t mean there isn’t a real need to add that kind of mechanism to the core of Umbraco, it’s just it wasn’t addressed then, or kind of since. [Add link to Part 3 here]
Ironically the presence and visibility of the dashboard, is the exact thing that places the thought in people’s mind ‘how do I add new things here?’
So where is this going?
Well, when I’m on my travels delivering sublime Umbraco Editor Training, or workshops helping Content and Developer teams work together to get the best out of Umbraco and at such reasonable rates too. I am fascinated by the tricks and workarounds that people who use a system everyday come to adopt, not because they’ve been told about them (often the reverse), but well, because they’ve discovered a way to bend the system to work for them, no harm done.
And it feels good if you find a way, 'that works', that’s not supposed to… and it’s a secret.
Like nudging in two identical fruits onto a win-line in a fruit machine, if the next spin you are offered to hold the reels - you don’t hold anything - let all the reels spin for a guaranteed win! It makes no sense, but it works and that feels good.
So the content team at this company were told to email any redirects they wanted to set up to the developers via IT, who explained they needed a bit of notice to set them up, and they’d add them to a rewrite map in IIS, but nobody could really remember what had already been set-up, it wasn’t easy to manage. (and a duplicate entry caused a ysod - which triggered the crisis discussion).
The redirects are content, editors need full control of them.
When I was explaining about redirect options in Umbraco, one editor put his hand up and said,
“ahh, but if you rename a page in Umbraco to the redirect you want, and then name it back again, then switch to the Redirect Url Management tab, and remove the one from the list you don’t want, then you’ll see Umbraco has created a 301 redirect for you - let the reels spin and you are guaranteed a win!”...
Have a look at what I am dubbing the rename-rename-remove redirect method, or rrrr for short.
.. .And next time I visited them, they were all doing it!
Life, uh, finds a way
(if I did giphys there would be a Jeff Goldblum one here right?, sorry to disappoint.)
The dashboard gave the visibility that enabled people to find a way, and people always will, find a way. And if they are doing this already, and you can’t stop them - then it becomes an editor journey in the CMS, ...and you perhaps are familiar from earlier tales in this blog, I’m all about reducing editor journey times and making tasks simpler… responsibly.
So I added a form to enable the editors to add redirects to the dashboard, but only to content in the cms that they could pick and only for relative Urls - so it was only a shortcut to repeating the ‘rename-rename-remove’ hack….
Here is the Rum Do version of the Redirect Url Management dashboard, same task:
A stopwatch task that had taken minutes to complete, and a bit of tomfoolery, was now down to under a minute, and it felt less arduous, less faff, especially for multiple redirects, it had removed the “do I really have to do this crazy thing, sigh” factor.
But is that it?
not quite...remember the second question in the discussion?
What redirects have already been setup for this page?
With the current Redirect Url Management dashboard you can search for redirects that have been set up, that’s because it’s designed to let you find errant redirects and remove them, but it doesn’t let you search by the target page, it doesn’t answer the question, what redirects actually exist for this page?, and if you have a content item open, why would you switch to the dashboard to search... if we've learnt one thing during these posts, it's that 'context switching' has a big impact on mental agility and how 'quick' a task feels to accomplish...
... so I added an action menu item - Manage Redirects, and now from within any content item in the content section you can click ‘Manage Redirects’ and a slidey-out panel will reveal a list of all the redirects from the Redirect Url Management dashboard for that particular page... without having to switch contexts...
...and then because at that stage I was a bit giddy with RUM (redirect url madness) I allowed the editors to create the redirects from the context of this slidey-out panel…
... same task - the journey is even quicker.,. but more importantly, it feels more intuitive...
I haven’t released this as a package, and I'm not sure whether it would be a helpful short-term addition to the core dashboard (it again is not trying to address the bigger picture of redirects in Umbraco)... what do people think?
...but in the meantime... should you work with editors who have mastered the ‘rename-rename-remove’ trick (rrrr), and you want to just let them do it easily...you can make use of this dashboard addition and slidey-out panel from the git repository here:
No, I’m not done... and is anything ever done? Each small change influences the experience and tramples over what next...
Embrace:
”ooh, what would be good if...”
In short a butterfly beating its wings in New Mexico can cause a hurricane in China, so who knows what global natural disasters have been triggered by the affects of our tinkerings, prototyping and implementing of the small changes I’ve been detailing in this ‘no longer a trilogy’ of blog posts...well hopefully it’s not just wind.
Keen followers of the series will know we are using light touches to speed up the editors’ ‘writing articles and picking images’ experience in Umbraco, as they switch and learn to adjust to Umbraco 7 - we want them to feel like the wind is behind them and they are not having to fight gusts of outrageous cms implementation. Umm It’s time to switch analogies:
I’m thinking that when you cook with an onion, perhaps you are making a Lasagne or French Onion Soup (see the recipes section of this site for further inspiration), there comes a point when you remove the first papery layers of the onion and are ready to slice and dice, but underneath there is perhaps revealed another wrinkly layer, or a strange black mark, and well, you don’t quite fancy it, and you peel off the next layer in the hope that underneath is something pure and unblemished that you can happily slice into... you may need to buy more onions to conjure your French Onion soup to perfection (p9 of the tooorangey cook book), but in your opinion it will be worth it to serve at your sophisticated dinner party later on - however if you are on a budget, well, the aim is perhaps simpler, to peel the layers. really, to just get to something that you don't think will kill you.
With software development similar levels of pragmatism can come into play when you consider project budget vs features vs time. If you have agreed to deliver a feature for a certain amount of money - it is open to interpretation when that feature has been delivered, you may work for much longer on that feature than you had considered, because as the feature begins to be realised, the client (and also the developer) starts to have/spark off ideas and sees what the potential of the feature could be. This can lead to a problem where the developer just wants to be finished, and get paid, and the client just wants the feature to be like they now in hindsight understand they would like it to be, and this isn’t the best atmosphere for creativity or fun.
Digression: Software development isn’t similar to the process of building a house, as people are often keen to suggest
- maybe it would be, if the thing you were building was something you and your client had had a lifetime of experience living in / visiting other people's houses to eat French Onion Soup in or even had just watched several episodes of Grand Designs back-to-back together on the telly so had some shared understanding of what was possible, oh and that the new eco light aluminium window frames from Germany might arrive late -
in good software, like cooking, you may follow a recipe but ultimately if it needs more salt, you don't argue the toss about the salt being out of scope for this phase of the soup...
development by definition is an event constituting a new stage in a changing situation.
So however you approach it, software will ‘develop’ incrementally, but how can you enable this so the client and developer know when a feature is done without falling out? - since pretty much everything you look at can be improved upon in some way? I sometimes wonder if development is just a seesaw of embarrassment between client and developer, and the development stops when the client is too embarrassed by the effort gone in to suggest any more changes, and the developer is not embarrassed by the quality of what they have shipped.
The first one to mention "Is that a showstopper, perhaps we could revisit this in Phase 2" is the first party to realise they never want to work with the other person again. The client changed their mind, the developer didn't understand.
I've talked at length before about managing expectations, and how under promising and over delivering is kind of much better than the other way around, and it's ok to 'not know'.
But there is another way to trigger a different atmosphere on a software development project, and that is to identify a common enemy, that the developer and client can unite against, and my favourite common enemy to use of late is time.
Having some kind of limiting reign over ‘what can be done’ within a software project may feel restrictive, but actually is really helpful for ensuring some pragmatism and stimulating clever and creative solutions, more importantly though it can keep the client - developer harmony intact, because they are ‘in it’ together, to try and get something delivered 'within the time'.
(Tip: Don't use money as the common enemy - nobody enjoys being told the don't have enough money, and people can always unexpectedly find extra money, and then people get upset if they don't get what they thought they were getting for their money - so use time, time is the best common enemy - it is not controversial and it's easy to understand… "the Dinner Party is at 7pm, I do not have time to peel twenty onions" - "we are effectively homeless next Tuesday unless the roof is on". (a waterproof covering of some devising is therefore much better here than half a perfect roof, or indeed the bowl of French Onion Soup the project plan said you would deliver on Tuesday).
The conversation becomes not “When will feature x be ready?” but “On x-day what features will we have?”.
Against the common enemy of time, it’s easy for the developer to show they are doing their best within the timeframe, and easy for the client to understand how long things take - particularly if you have a daily show and tell of the progress, and plot together against the common enemy.
It feels wrong, but once freed from the constraints of perceived perfection, and architecting 'the whole thing' you can start to enjoy working with your client comrade on delivering simpler smaller things quickly that focus on the immediate needs, stepping stones to where you want to ultimately get to, but with the flexibility to get somewhere else if it seems like a better place to go along the way.
"This is a lot of waffle...normally in these posts, we skim read and put up with some of the waffle, but then there is a link to some code we can sneak a look at and perhaps repurpose - when is that bit happening?"
Well, let me try to illustrate the above with an example.
We upgraded a magazine site to Umbraco 7, 250,000 media images, I’ve mentioned a bit about it in previous posts. After the upgrade two weeks were put aside for ‘other improvements’ - and the immediate concern they had was the speed of the ‘Uploading and Picking of Images’ journey, it was slow in Umbraco 6, and the upgrade hadn't really altered this perception.
The first thing to note here is we’re not ‘implementing a feature’, we’re improving a process, we’ve not agreed exactly ‘how’...
... we thought about architecting a new kind of Super Media Picker, building something from scratch to address any problems, perhaps working with examine, but in 2 weeks?
... instead we investigated:
Upload speed - at trade shows it was taking 12 minutes to upload an image, which was frustrating as the client were trying to cover ‘what was happening now’ as events unfolded... and not what happened 15 minutes ago. I immediately assumed this was something to do with mobile device connectivity at the event. So we set about proving this, I uploaded a file from my mobile, in an area of good reception and noted how long... umm it was 12 minutes - so I tried from my desktop, err 12 minutes - it was nothing to do with the shows! Back in the office they uploaded images in batches of 8+ to use within an article, so accepted this would be slow… and went off to make a cup of tea - they didn’t experience the single image upload problem until they were at a show.
This was weird, never seen nothing like this in an Umbraco site before...had we missed something in the upgrade?… we set up a vanilla install of their version of Umbraco, and pointed it at their database, and attempted to upload: 12 minutes again. The images were stored in blob storage... we turned off blob storage temporarily, 1 minute upload…. 250,000 images, would blob storage deteriorate with that number of images? (that’s not how the cloud is meant to work!). Darren stepped through the upload process, and found, the image upload part was actually really quick... but have you ever wondered where the number comes from in your media url: /media/12345/yourfile.jpg ? - well this was the cause of the stupid ten minute delay. It's calculated by looping through all the directories in the media folder and finding the one with the highest number:
_folderCounter = 1000; // seed
var directories = GetDirectories("");
foreach (var directory in directories)
{
long folderNumber;
if (long.TryParse(directory, out folderNumber) && folderNumber > _folderCounter)
_folderCounter = folderNumber;
}
This performs reasonably well for a file system on a server even for 250,000 folders, but in blob storage the performance is awful (blob storage is not designed to be queried in this way).
(The good news is now that media in Umbraco has a guid, the latest versions of Umbraco can use this in the calculation of the unique media url, and this iteration won’t need to occur.)
To work around this in the meantime, in the simplest way, we created a hybrid File System Provider, that used Blob storage for storage, and the file system, for the calculation of the next number.
So the nice thing here was it was really easy for the client to understand the problem, we showed them the code above, and they could get why it might be slow with 250,000 images.
Did this mean the File Upload and Media Picking journey was quick enough?
No, we had just peeled off one layer of the onion, the client could see some more blemishes.
Picking Images
Images were now uploaded in a comparatively astonishingly quick time but ‘picking them’ was still slow..
We looked at the Media Picker and found, it was slow, the client had tons of images arranged in Alphanumeric folders, and subfolders eg
[Article Images] - [B] - [BR]
And each click along the breadcrumb to drilldown to the images gave no indication that the media items were loading, encouraging further 'impatient from waiting' editor clicks, resetting the request and slowing things down further.
We added a loading indicator, so at least the editor knew to wait…
(this is now coincidentally been added to the core in 7.6.x)
This meant that although it was no quicker, it felt quicker, because the editor had feedback something was happening. (My favourite kind of small fix).
But why was it slow on each folder request?, we found that all the images for a folder were being requested without paging (although this looks to have been addressed in v7.7)... for each folder click, there could be hundreds of images, and for each image, along with the generation of a thumbnail, all of the Media Type's properties were being returned (when the media picker perhaps only really needs Image, Name and Id?). This particular site compounded the problem though - in an attempt to use the Media Section as some kind of Media Library the default Media Type Image had an additional 23 properties (instead of just width, height, size type etc)
.
Each of these additional properties were being retrieved in the blob of Json for the Media Picker to display the images to pick (all via the MediaService, eg db requests), this additional unrequired data on the request was overwhelming the browser (Over 2mb of Json)...
...again we were thinking, stop, new Media Picker + Examine/Azure Search, but again a bigger task than the common enemy time would allow.
Understanding this the client agreed we could drop some of the media type properties, (you can see from the screenshot they weren't filled in very often), and we figured we could scale up the database a notch in Azure, it would cost more but the trade off was faster picking, it would be worth it, but whilst looking at the queries that were being executed, Tom Pipe had a hunch that the database wasn't performing as quickly as he'd expect (it was already scaled up, it shouldn't be this slow).
Digging into SQL he found a thing, which we all remember from the old days, but as we're not DBAs anymore we'd collectively forgotten about:
Heavily fragmented SQL indexes
(And we’ve since found this on many of the Umbraco sites we support, particularly those upgraded from 6 to 7)
The advice from Microsoft is to rebuild your sql indexes if they are over 30% fragmented, and reorganize if they are greater than 5% but less than 30%.
Tom being Tom, built a custom healthcheck to test for fragmentation and allow you to rebuild/reindex as necessary from the backoffice:
Which he says he's definitely going to release, so why not remind him once a day on twitter: @toepimp until he does.
Anyway you can see the result of rebuilding the indexes on database performance generally, and the speeding up of the MediaService Api’s GetChildren:
We didn't need to scale up the db, in fact two weeks later we scaled it down a notch!
So are we there yet?
Has the final layer of the onion been peeled?
… not quite.
Media picker in a Macro
One editor was reporting the media picker was slow, when used via a Macro to insert an image into the Rich Text Area. What could be different about using the Media Picker as a Macro Parameter we thought? We investigated and found no difference, we arranged a screenshare for the editor to demonstrate and the bottleneck was obvious:
When the editor hit the Insert Macro button, the macro selection dropdown appeared:
However the site only had one Macro! the slowness being reported by the editor was caused by the obvious frustration of having to pick the macro from a list of 1 in order to get to the Parameter Options that included the Media Picker... It felt clunky and slow, why doesn’t the insert Macro option go straight to the parameter options if they exist and it’s the only Macro defined on the site?
My first thought was shall we add a second Macro to insert something else, so it doesn't feel so pointless making the selection :-)
...but instead the workaround here was a new TinyMce Custom Plugin, that fired the opening of the Insert RTE Image, (and then the actual Media Picker overlay) so the editor had a quick way of getting to the insert image part, when they wanted to insert the image instead of faffing around with pointless dropdowns of annoyance making the picking process feel slow.
So tell me we can start cooking this Onion Soup now?
Nearly there...ready to peel another layer?
Media Picker List View
The final issue the editor reported having difficulties was an interesting one, essentially their articles might contain seven or eight images, all focussed on a specific technical aspect, and typically the images would be named:
Long and complicated feature picture 1.jpg Long and complicated feature picture 2.jpg Long and complicated feature picture 3.jpg
Which they would upload as a batch to the media section or on first pick via the media picker.
The problem here is that the images all sort of look a bit alike, with subtle variations, the media picker presents images to be picked, as umm thumbnails of those images, the editor knew the filename to insert where, but could not necessarily pick this image out of an identity parade. But you can hover over the image in the media picker and reveal it’s filename right? yes but..
Two problems, firstly you only see the first part of the name, so the distinguishing bit ...picture 3.jpg or ...picture 4.jpg was cut off - and secondly the images aren’t displayed in alphabetical order - their order is determined to best fit the image thumbnails into the picker overlay - and so you’d be hovering and guessing between each one trying to guess, if the next one, was the one, waiting for the hover to kick in.
I couldn’t really determine if this is a really niche problem or something that affects editors more generally, how common is it? it came up in conversation from the client that you’d think you’d be able to switch the picker to use a List View like in the Media Section.
And I'm a great believer in 'what you would expect to be able to do' to be perhaps what you should be able to do...
... but to do so would mean custom media picker, which we didn't have time to create, or could we tread lightly upon the existing picker?
intrigued, I started with the existing Media Picker, views and controllers and directives, and I created a prototype custom media list picker to explore the idea - attempting to change as little as possible, essentially creating only a custom Media List Directive to replace the umb-media-grid directive, so it had the same functionality as the grid view, but with a toggle to switch to a list view.
At any point in the media picking journey you can switch to see the ListView version, and the names of the images appear in full, and most importantly in alphabetical order, which is exactly what this editor needed. It speeds up their picking of media jouney when they don't know exactly what the image looks like, and less mistakes are made - anecdotally it's just less frustration, so again feels quicker.
It isn’t really something that will work as a package, as the Media Picker evolves in the core, you’d have to keep evolving this, but if more editors might benefit from this feature, and it would be great to hear if people think this is a good idea - you could imagine it being added to the core Media Picker as an option?
Have added an issue to the issue tracker to gauge any interest.
It’s a prototype, the editors are using it on the site, and feeding back (could we hover over the list item and see the image etc. can we have an option that shows the last 20 uploaded images - we always pick what we’ve just uploaded etc)
This small change is already sparking off different ideas for future improvements, and I know I ended up creating a custom media picker in the end - but it's very different to the one I would have architected at the start of the two weeks, and that prescribed version probably wouldn't have addressed any of the actual issues above, although it would have used examine/Azure search, and been ace in different ways, it would have been problematic in others...
Does this illustrate what I'm trying to say?
Peeling away the onion layer by layer, working with the client, beating the common enemy of the two week timebox to deliver something useful? The difference is the client is eager to do more, we're not arguing over the over-promised "Super Media Picker" being late, and that it turns out to also feel slow within a Macro, because of the dropdown... etc etc
Summary: If you want to choose a trade based analogy for software developers, I don't think that we're builders, nor architects, we are much more akin to gardeners, working with and against the changing seasons, we have experience, we know certain plants won't grow well in that soil, we work with the client, it's their garden, we advise, reflect on what's happened, each time we come, we listen, plan and improve the garden, but it's never finished.
“... and then we publish from Umbraco and one final check, the blog post is there, and we are done”
“Great, I’ll stop the watch, how do you celebrate? A nice cup of tea?”
“.. yes, no, well not quite yet, I’ve just got to tweet this out, it’s a pain, I’ve got to logout of my twitter, log into the twitter account for this blog, cut and paste the url, compose the tweet, tweet it, then remember to logout of the company account and back into my twitter, else I’ll be posting pictures of my lunch on the company blog account, which I actually did once by accident”
“A-ha, I’ll keep the stopwatch running then”.
So I’ve found myself writing what has become a series of blog posts, dubbed, in some of the most unfashionable areas of the internet as the ‘working with editors, not against them trilogy’, and they are all following a common theme - the short recap of which is, at Moriyama we’re helping several groups of editors in different companies adapt their editing workflows within their freshly upgraded shiny Umbraco 7 environments, and at the same time becoming slightly fascinated with the human interactions, reactions and thought processes involved.
Today, I’m kind of highlighting a specific nuance of the flick-flack of editor-developer interaction that is sooo subtle, it might not even exist, so bear with me.
In the illustration above, the editor is expressing a specific problem, “the switching of twitter accounts between personal and work accounts”, to send a company tweet about the blog post they created.
Interestingly earlier in the project one of the backlog items had been a vague creation of a ‘complete social media sharing platform™’ which had been sized as a fairly large T-Shirt(XL) and been dropped out of the scope for the set of ‘upgrade’ sprints.
As a developer though, given a real problem, you immediately start to think of how you might solve it, and as you read this you are probably thinking yourself of a number of different really neat solutions... but when handed a ‘specified solution’, for example: in the case of the ‘complete social media sharing platform™’, you’ve got nothing to go on… you haven’t got the ‘Why?’, there is no spark of an idea, the problem solving part of your brain isn’t triggered… and consequently, more importantly, your enthusiasm has evaporated, it already feels like a slog, and that’s just reading about it here and imagining you might have to implement it one day.
If we’re honest, one of the main drivers for a developer is 'to be the person who solved the problem', you take more ownership of the solution, it’s your idea at stake, so you know you might even ‘test it’ before you show it off on the client demo… it's a giddy exciting feeling when you've made a difference.
I’m not a great fan of 'ego driven' development because it tramples over teamwork (and my teamworks is the best! much better than yours) but there does appear to be some truth in the maxim:
present developers with problems to solve, not solutions to implement.
(Notice I’ve pluralised developers) - I think you end up with better solutions, but even if you don’t, everyone appears a lot happier and excited working on the thing, when it feels like it's 'theirs' - albeit I have no empirical data to prove this.
I said it was subtle, and might not actually exist.
Empathy
The other interesting thing here is the dissonance between what ‘is Umbraco’ and what isn’t… the tweeting part of the journey was perceived as a ‘thing not worth mentioning’, because it was outside of Umbraco, but is actually still a really key part of the whole editor journey. I only seem to uncover (or stumble upon) this kind of information, tangentially, when in conversation with an editor, perhaps during a training/orientation session or in a show and tell progress demo call, I think in those situations it's easier to be empathetic - however it's hard to sell the concept to the client that really they should pay for a developer to 'just watch them’, that's apparently 'a bit too creepy'....
Anyway, Sorry I’m doing that thing of going on about stuff I'm not sufficiently professionally trained to wonder upon publically and you just want me to get to the bit about what we did… and link to some sort of code I must have hacked together.
What we did…
Ok, so the crux of the problem here is the company blog twitter account is owned by the blog - not by the individual, so what can we do about that?
With this installed we can then add an ‘Authenticated Twitter Account’ option on the blog homepage: using the Twitter OAuth, DataType that comes with the SkyBrud package.
You’ll also need to create an ‘app’ with Twitter, to tell them that you are about to start programmatically authenticating and doing stuff with the API, it’s easy to do here: https://apps.twitter.com/ and they’ll give you a ‘Consumer Key’ and ‘Consumer Secret’ that you’ll need to cut and paste into the configuration for the SkyBrud Twitter OAuth DataType.
And if we’re logged into Twitter with the account we want to authorize, we just press the ‘authorize button’ and accept the following message:
And if you authorise the app, you now have an Authenticated Twitter Account associated with your blog:
So great, can we tweet every time something is published?
No! - I mean you could but I think you have confused a twitter account with an RSS Feed.. (and there are services that will schedule tweets from an RSS feed) - tweets should be mildly interesting, have some personality and be written for engagement, in my opinion...
… but we DO want to shave some time off that 'logging out logging in cutting and pasting' journey discussed earlier, how best to send a tweet from here that isn’t automated?
How about a Custom Menu Item?
We can make it only appear on the actions menu when a blog post is loaded and the editor can choose when to tweet
To add this custom menu item you have to write some c# code and hook into the Umbraco Event that fires when a Menu is rendered:
Create a class called ‘RegisterEvents’ that inherits from ‘ApplicationEventHandler’ (or maybe your project already has one) we need to override the ApplicationStarting to add the hook to handle the event:
Tip: Type TreeControllerBase.MenuRendering
and then type: + = Tab Tab
It’s an exciting Visual Studio shortcut combination that will stub out the handler for the event and you should have:
public class RegisterEvents : ApplicationEventHandler
{
protected override void ApplicationStarting(UmbracoApplicationBase umbracoApplication, ApplicationContext applicationContext)
{
//register custom menu item in the media tree
TreeControllerBase.MenuRendering += TreeControllerBase_MenuRendering;
}
private void TreeControllerBase_MenuRendering(TreeControllerBase sender, MenuRenderingEventArgs e)
{ //implementation here
}
}
This is great trick if you know the name of the event but there isn’t clear documentation of the event signature to hand…
Anyway the code to add the menu item will look something like this, I’ve commented what’s going on.
// we only want to add this option to the content menu
if (sender.TreeAlias == "content")
{
// get details of the content item
var umbracoHelper = new UmbracoUmbracoWeb.UmbracoHelper(UmbracoUmbracoWeb.UmbracoContext.Current);
var nodeId = e.NodeId;
var contentItem = umbracoHelper.TypedContent(nodeId);
// if the content item is a blog post
if (contentItem != null && contentItem.DocumentTypeAlias == "BlogPostAlias")
{
// create the menu item called ‘Tweet This’
var tweetThisMenuItem = new UmbracoUmbracoWeb.Models.Trees.MenuItem("tweetThis", "Tweet This");
//assign an icon
tweetThisMenuItem.Icon = "bird";
//separate it from other menu items:
tweetThisMenuItem.SeperatorBefore = true;
// now this is where we wire up the functionality of what to occur when the menu item is pressed - we tell the menu item to open a slidey out panel (actionView) and load the ‘tweetthis.html’ angularJS view into it:
tweetThisMenuItem.AdditionalData.Add("actionView", "/app_plugins/tooorangey.TweetThis/tweetthis.html");
// insert it somewhere in the list of existing menu options
e.Menu.Items.Insert(5, tweetThisMenuItem);
}
}
Great we now have a custom menu item, that appears on a blog post, and when you click it, causes a 404 error on the missing tweetthis view!
We need to create the view, and because we’ll be wanting to add some further functionality we also need to create an angularJS controller and a stylesheet, and finally a package.manifest to tell Umbraco to load these assets whenever the view is loaded:
The package manifest just needs to reference the js and the css assets:
So now when the editor clicks the 'Tweet This' menu option, our custom view, the TweetThis.html is loaded into the action panel, and the package manifest file (simply because it's in the same folder as the custom view, loads the angularJS controller and css we need for our Tweet This Textbox).
The textbox measures the number of characters and is preloaded with some suggested text, eg the title of the blog post, and a link to the blog post, (and the template for this default text can be tweaked via a property on the blog homepage)
Which produces something like this:
Notice how it's highlighted 'who you are tweeting as'!
The counting of the number of characters left in a Tweet isn't as straightforward as you would think, all links, in tweets are shortened using t.co, so you can’t just count characters, so you have to adjust the total remaining to allow for this, so any urls entered > 23 chars will not use up anymore of your allowed 140. (23 is the maximum length of a shortened t.co link in twitter, there is an api /help/configuration that will tell you whenever this number changes, but of course I’ve hardcoded this for now, as I like ticking time bombs)
Now when the editor clicks Tweet This, their message is tweeted!
How do we send the tweet?
We have a TweetThis action in an api controller (UmbracoAuthorisedApiController) that the text of the tweet is sent to and again SkyBrud does the hard part:
public IHttpActionResult TweetThis(TweetInstruction tweetInstruction)
{
try
{
//read authenticated twitter account details
var twitterAccount = getTwitterAccount(tweetInstruction.ContentId);
if (twitterAccount != null)
{
//get twitter service using authenticated twitter details
TwitterService service = twitterAccount.GetService();
TwitterStatusMessageResponse response = service.Statuses.PostStatusMessage(tweetInstruction.Message);
return Ok();
}
}
catch (Exception ex)
{
LogHelper.Error(typeof(TweetThisApiController), "Error Tweeting Message", ex);
}
return BadRequest("Error");
}
And here is a successful tweet...
notice we’ve implemented 'Twitter Cards' on the site, so whenever the blog post is shared it get's a preview with associated article image.
But it's not my Twitter account - what ‘has’ been tweeted out recently?
Because this isn’t the editors usual twitter account, and on some blogs it’s shared with other editors we’ve kind of defeated the object of this improvement if the editor still has to remember the twitter account, switch to twitter, read the recent tweets, make sure they are not duplicating previous tweet content or style, then switch back to Umbraco to make the tweet. So now that we’ve got Skybrud setup and the Blog Twitter Account authenticated, it’s possible to list the recent tweets out via a custom property editor on each blog’s homepage perhaps on the ‘Social Tab’ in the backoffice:
So what’s the effect on the journey stopwatch?
It's hard to compare fairly the two complete journeys, since a certain amount of time is spent thinking about and composing the tweet, but if we consider 'time until point of composure', It takes a few minutes to switch twitter accounts, so the obvious gain, here is being able to click Tweet This and in 12 seconds you are at the point of composure. Having the 'default text' in place saves further time, but hard to assess, whether this affects the quality of the tweet message, eg does the editor have more time to think about the actual message, or is the default 'good enough' - I'm going to pretend it potentially does both, and focus on the fact that 'time to the celebratory cup of tea' is drastically reduced.
So small little tailored changes like this, might seem unimportant or be brandished as 'gold plating' but any task that involves faff, even if the duration of the task isn't very long, wears us down, and it's particularly draining when you have to switch context or systems, whilst keeping the original task held in your mind, so tiny improvements of this nature can have a big impact on editor happiness and their outlook on an upgraded implementation.
"Umbraco 7 is great, you can tweet from the blog editor page"
Anyway you can have a closer look at the moving parts in this gist of a git repository, if you are interested in the implementation details.
So I think as I explained before in an earlier post, here at Moriyama we've been working recently on 2-3 coincidentally similar projects, involving upgrading super-large Umbraco 6 sites to Umbraco 7 and supporting the editorial teams in making that switch and adapting to their new editing environment, and I think we've established that it's the human element of an editor experiencing Umbraco 7 for the first time, from the context of years of Umbraco 6 ‘enslavery’ which has really peaked my interest. Well, that and making musical based puns on any solutions that 'make it' into the community as Umbraco packages.
When something changes that fundamentally affects how you do your day to day job (and this is magnified if the change is thrust upon you), there appears to me, to be series of ‘states of mind’ that an individual needs to pass through, before they successfully adapt to the change, until they stop calling it the ‘new’ system - a bit like the seven stages of grief.
During this journey of adapting to change there appears to be a common state of mind when an individual will ‘question everything’, which is probably a really common part of human nature, mistrusting something new is a good survival technique.
This can feel really tiring, almost like the person doesn’t want the new system to be better, but if you are tasked with introducing the new way of working to that person, avoid being defensive: patience and explanations of ‘why’, and lots of patience are required to get through to the next stage - do not tell them the new thing IS better, instead, respect them to come to their own decision given time and information, and learn yourself from their feedback and how they see the change with their fresh grumpy eyes.
Anyway the interesting thing about a person in this mindset is they can often stumble across a series of steps or contrive a situation to find a bug or a flaw, that can take you by surprise…
… let me give you an example.
Take the journey of creating a new document and updating it in Umbraco 7 - saving it as you go along.
You and I know - it's not possible to lose your editing work in that scenario, is it?
you've pressed Save, it saves, right? - Umbraco is pretty reliable at that, and we know if we click away to another part of Umbraco we'll get the:
Discard / Stay option
If we're honest, we get that message a little bit too often than we'd like, it's annoying, but we know it's trying its best to help us, so we tut a bit, but don’t begrudge its existence.
So clicking the wrong option here is not what the user did to lose their work…
(notice how the colours have changed in 7.5+ so the ‘safe option’ to stay is now highlighted in green - so tired editors don't accidentally click the previously emphasised 'Discard' option (often semi-consciously, when tired or in a hurry you will inadvertently click the highlighted option on a dialog, without really thinking))
... wait they lost their work? Yes, here's how, why not follow along at home?
Create a new document in Umbraco 7
Perhaps a blogpost
Type some content into the document, press Save, type some more, press Save, type some more press Save, etc now after your last Save, click away to another part of the Umbraco content tree, you don’t get the 'Discard / Stay' option, now try and get back to the article you were creating…. It has gone!
What? How?
Now if you were following along at home you might have without thinking added a ‘Name’ for the document at the top, but if you did not, then when you press save, the feedback to say that this is required, is really subtle, (just a red border around the name, not consistent with the rest of the validation error messages).
If you are coming from Umbraco 6 you are a little bit blind to this area of the screen. So when you press save, your document isn’t saved, but that’s kind of ok, because the Name hasn’t been filled in, and you can’t save without a ‘Name’ dummy! - it's just you won't notice the absence of a save confirmation if you are not expecting one - but here’s the bug - pressing Save clears all the angularJS IsDirty() flags, whether the document is saved or not!!!! And this enables you to click away without the discard warning… and then as it’s not saved anywhere, you lose the content you just entered!
The subtle difference is in Umbraco 6, you had to supply the name at the point of creation, whereas in Umbraco 7 you can start editing the document without it having a name.
So the answer to this is to just explain to the editor to fill in the Name at the top of a document before they start creating content - but for someone transitioning to Umbraco 7 from 6, this occurrence is frustrating and saps confidence in the ‘new system’.
“It’s worse, I never lost content like this in Umbraco 6”
It increases the length of the phase the editor is in of ‘mistrusting the change to the new cms’, they question everything more and more. It harms their productivity.
Here’s what we did.
Thanked the editor for discovering this - where were you two years ago? When people were testing this, bet this has driven other editors mad over the last two years and nobody understood why - now you’ve pinpointed it!
Raised this as an issue: http://issues.umbraco.org/issue/U4-10108 and explained how reported issues get fixed and improve the product for everyone, look at all the issues reported, it won’t be fixed immediately.
Provide a short term quick workaround that acknowledges the seriousness of the issue to the editor…
Short term fix
So if someone forgets to type into the Name box (as it’s outside the editor main screen and they are not used to having to fill this in) let’s make the assumption, that they will type into the ‘Title’ box, the first property on the document type for this site.
Let’s create a custom property editor for the text box and have anything entered into the Title textbox ‘sync’ into the Name box on 'first load'...
...(but not after it’s been saved once that would be annoying) - now it will be impossible to type a title and not have a corresponding name for the document, we can make the title a ‘required’ property, and now no work can be created and lost accidentally in the weird scenario outlined above.
The code is here on github if you are interested. No musical pun. (‘Sync me up before you go go’ would be very poor and even below my standards - yes! - I have standards),
Editor is happier, we have listened, acknowledged the seriousness and demonstrated how quickly we can respond to issues like this in Umbraco 7 and Opensource.
Fast-forward to one week later, follow up visit and the editor says:
“Thanks for that sync box thing, but I’ve just learned to type a Name in first, so we don’t really need it”
Result! - the editor has adapted!, but in a positive way, it feels like we were on their side all along - they are sharing their success with us, the custom property editor of course wasn’t the ultimate solution, just temporary stabilisers, to give the editor confidence and enabled them to move on to the next phase of adapting.
Admittedly I don’t have a ‘scientific control editor’ that we told to ‘just put a name in first’ - this is all spurious and anecdotal, but I mention it really to try and highlight ‘there is a lot more going on’ when people use systems for the first time, or things change, and often amongst fellow developers I get a vibe, that editors are somehow at fault, but we should recognise it’s not a binary or even linear curve that humans follow when they encounter change, and if you support editors through the transition phase, the long term outcomes will be more positive.
The developers job doesn’t stop when the code is shipped.
We have been working on two upgrade projects from Umbraco 6 to Umbraco 7 this month, and it’s thrown up lots of interesting responses from the editors involved in making the move and adapting to their new environment.
“It’s just the same with a different font”.... was my favourite grumpy observation.
You get a bit complacent knowing how much better Umbraco 7 is in terms of editor experience that you take for granted that editors will embrace the change, forgetting that if you’ve used a tool for 3-4 years, and change is forced upon you, there will be at least some resistance, it’s human nature.
But for me it’s a really interesting scenario, and the first thing you do is stop telling people how much better Umbraco 7 is to Umbraco 6, and you listen - and you do this because editors use the product every day to edit content and they are 'seeing with fresh eyes', it's gold-dust and seconds and minutes within a particular workflow are incredibly important to them - they are the experts.
Media Picking
Take Media Picking for example: On one site when we sat and benchmarked editor activities for picking media for articles in Umbraco 6, it was monumentally slow, either an editor needed to switch to the media section to upload their image, and switch back to the content section to pick it or they used Digibiz Advanced Media Picker DAMP - that enabled them to upload to the media section from within the content section and set particular crops - but was really really slow.
“You’ll love Umbraco 7 for picking media", we boasted, "it’s one of the biggest improvements, it’s really fast and you can upload and pick media from within the content section, no more switching between the two, your workflow will be faster”
In both scenarios, we replaced with a standard Umbraco 7 Media Picker - and both workflows when re-measured using Umbraco 7 were faster: (stopwatch method - complete the journey as fast as you can).
Switching between Content and Media Section to upload image and then pick:
Umbraco 6: 3 mins 44 secs
Umbraco 7: 1 min 3 secs
Upload within the Content Section and pick:
Umbraco 6 - Upload via DAMP: 1mins 24secs
Umbraco 7 - Upload via media picker: 35 secs
Upload within the Content Section, pick and define crops:
Umbraco 6 - Upload and Crop via DAMP: 2min 21secs
Umbraco 7 - Upload via media picker: 1 min 42 secs
So basically it’s a win all round right?
Well no…
...all is not as it seems.
The gains of using the Media Picker to upload an image and pick selected crops is not ‘that much faster’ than using DAMP - this is mainly because with DAMP, you can edit the crops without leaving the content section in a popup window...
...even though doing so is slower in actual terms of task time, it feels much quicker ‘mentally’ because you haven’t had to shift your focus to another screen, switch context and switch back again.
The editors reported the journey in Umbraco 7 as feeling slower.
Also if you were in the middle of editing content at the time, clicking the pencil on the Umbraco 7 media picker to open the image in the media section to enable the crops, will have triggered the ‘you have unsaved changes’ dialog, which will cause you to think, jolting your mental process - and you’ll have to save your content item, switch to the media section, adjust your crops, and switch back to content to complete the journey.
The paradox is this is still actually quicker than using DAMP! - but it is the exact workflow you boasted earlier that Umbraco 7 had solved!
Essentially the excellent Image Crop facility of Umbraco 7 is not welcomed with open arms by this set of editors because it has broken their workflow!
Even though in real actual time the speed of the Umbraco 7 backoffice makes the workflow quicker to complete - It exhausts the editor in terms of concentration, and this is why it feels slow and for me, this is the really interesting thing about analysing UX/UI, and measuring improvements, is that you can’t ever discount ‘how it feels’ to the person, even if the stats say otherwise!! (think of it as the windchill factor that makes a cold day feel colder than it actually is)
Anyway Tom Pipe (@toepimp) and I figured that surely it must be possible to bring the existing cropping functionality from the media section into an overlay in the content section:
This is only a prototype, but very usable - we’ve taken the approach to have a new property editor that targets a media picker by alias and adds an Edit Crop button underneath the picker, clunky perhaps, but pragmatic...
... well we don’t want to end up maintaining our own custom media picker - we can release this for testing quickly and immediately improve the editing experience for our customer, improving over time with their feedback.
Hopefully, this illustrates the issue too, and a kind of resolution, and we can use this as the basis to explain the problem to Umbraco HQ that might lead to a PR or maybe it’s a bit too niche for the core, and it’s better as a separate package.
in which case what on earth would we call it?
Moriyama.ClubCropicana ???
(Darren IS on holiday this week)
or more sensibly:
OutCrop ?
Anyway, It reassures the client that we are listening, and that the upgrade puts them on a platform that quick improvements like this can be made easily, but hang on forget the platitudes! - what has it done to the stopwatch test?
Upload within the Content Section, pick and define crops:
Umbraco 7 with Moriyama.ClubCropicana/OutCrop: 53secs !!
uSpinMeRightRoundBabyRightRoundLikeARecordBabyRightRoundRoundRound or
uSpinMeRightRound for short… is a package that adds the functionality to rotate the orientation of an image once it has been uploaded into the Umbraco Media Section.
You can see a video here that shows it in action:
Why the plugin?
You’ve probably not noticed that a 'rotate option' is actually missing from the back office of Umbraco, this is because uploaded images are ‘auto-rotated’ to be the right way up, based on their exif data etc I think, and using Image Processors AutoRotate functionality... so why would an editor ever need to rotate an image in the back office?
Well if you are bulk uploading images it seems like sometimes the auto-rotate doesn’t kick in and also if you are uploading scanned images from a flatbed scanner like in the olden days then there isn’t the orientation data to auto-rotate by, or perhaps if you are like me and could never fathom how to put the document around the right way, (same with manually doing double-sided printing).
Anyway if you are cohabiting that Venn diagram overlap of niche circumstances then it’s super helpful to have a rotate option in the back office, and it saves you from downloading, spinning it around in some other image manipulator and having to re-upload the corrected version.
How do I work it?
Choose 'rotate' from the action menu
See preview of options for rotating the selected image in 90-degree turns
Select rotation option and click rotate…
(you can replace the crow spinning image at /app_plugins/tooorangey.uSpinMeRightRound/spinner.png)
Correctly orientated image.
And that is it. (although there is an option to create a new media item for the rotated image if you so desire)
How does it work?
This is, of course, standing on the giant shoulders of James M South and Image processor that ships with Umbraco now. Image Processor has a 'rotate' option built-in (a wrapper around System.Drawing) that is of course super-easy to use. However in generating the preview images of how the image will look in the various rotated states, I hit a problem in that I wanted to just add ?rotate= on the querystring, but the rotate processor is turned off by default (for good reason) in Umbraco and I didn't want to enable it for all 360 degrees for all images on the front end of people's websites just to do the previews for these three rotated positions in the plugin.
So I created a new Image Graphics Processor called ‘pirouette’ that will only accept a number of quarter-turns, 1, 2 or 3.. (I'm aware that pirouette means one full turn doh!)
Again, nods to Mr M South as this is just as simple as creating a class that implements his IGraphicsProcessor interface.
public class Pirouette : IWebGraphicsProcessor
{
/// The regular expression to search strings for.
/// </summary>
private static readonly Regex QueryRegex = new Regex(@"pirouette=[1|2|3]", RegexOptions.Compiled);
//etc
and add details of the processor to the processing.config file:
(It’s only since Umbraco 7.5 that the additional Image Processor configs have been shipped, so if you want to use this package pre-Umbraco 7.5 you’ll need to install the Image Processor config NuGet package – here…..)
Rotating the image
The actual rotating and resaving of the image back into Umbraco is done via a back office only UmbracoAuthorizedApiController, when you select the number of turns in the UI this instruction is sent to the Api, and we use Image Processors' ImageFactory class to perform the rotation. There is a save method that enables you to save the new transformed image into a memory stream, which is just what Umbraco's media file service likes to get hold of to add images to wherever it is Umbraco is configured to store them. So I have a feeling this will work in the cloud…
var imageToRotate = imageFactory.Load(fullPath);
var ms = new MemoryStream();
imageToRotate.Rotate(turns * 90).Save(ms);
But the tricky part here was trying to avoid file locks, eg what if the image you are trying to rotate is being displayed by IIS on site and locked - I think I've worked around this by always renaming the file with –rotatedX on the end of the filename, where X is the number of turns, this avoids the locks, and sort of also means if I’m doing something really bad here I haven’t mucked up the original image!
(If your having one of those days, perhaps your Umbraco site is powering an art gallery of conceptional art and it’s not clear which way around the picture should be anyway, and you keep rotating and rotating the image around in circles, then if the rotation for that number of turns already exists on disc, uSpinMeRightRound will use that instead for the media item.)
I’ve just realised in typing this that there is probably an edge case here if you have the text '–rotated1(2or3)' naturally somewhere in the name of your image file – well I’m sure it will be fine…
Finally, there is an option to create a new media item inside Umbraco with the rotated image keeping the original in place – the trick here is to make Umbraco think that the memory stream is an uploaded file by implementing a wrapper class that inherits from UploadedFile base class.
public class MemoryStreamPostedFile : HttpPostedFileBase
{
public MemoryStreamPostedFile(MemoryStream ms, string fileName)
{
this.ContentLength = (int)ms.Length;
this.FileName = fileName;
this.InputStream = ms;
}
//
and well
Anyway, it’s a first-release, there are improvements to be made (image selecting UI and making text translatable, but let's see if it’s of any use to anyone first).
Found this in a notepad file, as a possible ending for last year's codegarden talk... I think I know why I didn't go down this route as it would have been hard to pull off, but the bits that are very lemony snickett made me smile nearly a year on... so they might have worked:
Possible Ending ?
So I told the story of the super multiple checkbox carcrash from earlier to Benjamin Howarth at last year’s codegarden, and he said, ahh yes, I’ve seen that site, it’s xxxx and I said umm no, but realised the site he referred to was in the same industry, and so maybe this same person is using this pattern in all their development sites; and maybe this one person is responsible for all the bad code I’ve had to rescue in all Umbraco sites ever; I mean everyone I talk to about carcrash Umbraco has never done anything stupid; so surely it’s most likely one single person responsible for all this crap - everyone has a nemesis right ? - and no I don’t have black outs it isn’t a fight club thing - If only we could track this person down, I would gladly pay for them to attend the Umbraco training courses, and introduce them to Our Umbraco, and a subscription to Umbraco TV; there never need be a future carcrash umbraco talk again.
so Ladies and Gentlemen I can reveal I have dedicated the last year and a half of my life to tracking down this individual, I have gone undercover, grown a beard and walked and developed amongst you; I’ve travelled, by sea, rail and hot air balloon - slept rough in a ditch outside Byte5 offices in Germany - worked in Amsterdam bulb packing factory that overlooks Novaware HQ - played poker in Huddersfield with a robot, I suspected at the time to have been built by Matt Brailsford’ and I’ve answered countless forum threads obliquely, with leading follow up questions, and data mined tweets mentioning @umbraco (awesome is the top keyword), searching for clues, I have been like some kind of Umbraco **** hunter… to the extent that Channel 5/Discovery Channel would like to make a documentary about me...
.. and today today codegarden I can reveal I suspect the person responsible for all of these carcrash Umbraco problems is present in this room and I have concocted an elaborate scheme for identifying this individual.
(I don't know what the elaborate scheme was going to be)
Ok there is a big slab of a gap on the homepage... maybe on an intranet and the client says "oh that will be an RSS feed of the latest news or weather or something…" - because they want the page to be some kind of 'hub' for all information on the internet that might be of passing interest to their users/employees etc - although of course that isn’t how people use the internet is it?
What's really going on here is you have a big white space next to the company Twitter feed... And well, the output from an RSS feed would be about the same size.
Anyway, after sighing inwardly, you dig out that code from 2012 and if you were smart back then, you would have stumbled across the SyndicationFeed class from that then version of the .Net framework and you'll do the same old thing you always do to display the feed.
In Umbraco then, you're left with a dilemma, this is code, c# code, it shouldn't be in a razor view, although it's only a few lines? should I? should it be a Surface Controller Child Action - should I hijack the route? or....
... why not a property value converter?
is that really the right answer?
The use case
This package / custom property editor thing came from an editor on a site we support needing to update the RSS feed Url on their website homepage, they found the new RSS feed link, clicked it, saw the raw feed in their browser. satisfied, they copied the url from their address bar, and pasted it into the RSS feed textbox property in Umbraco, on their homepage node - published and boom... the url was missing the last few characters, bad cut-and-paste! ... their homepage was down!
Wouldn't it have been great if they could have checked the feed Url was working before they published the homepage?
Now they can!
Replace the textbox for the RSS Feed Url in the umbraco document type with a property editor of type tooorangey.RssFeedUrl.
Now after the editor has cut-and-paste the feed Url, they have a Check Feed Url button to press.
If there is a problem with the feed, they are alerted before the homepage is republished, before the boom!
and can check and correct the feed Url accordingly...
If the Url is an active RSS Feed, the editor sees a tick and a reassuring 'Valid Feed Url' message! - ahh but is it the right feed? a Preview button enables the editor to see the current contents of the feed, to check, again before they push the publish button.
Those first two items are not fake news as far as I know.
I'm not doing anything super clever to validate the Rss Feed, I'm just seeing whether the SyndicationFeed class can open and read the url and whether there are any feed items.
(Why doesn't the editor use Umbraco's page 'Preview' button before publishing instead...? well you are less likely to use this button for changes to an existing page, and also less likely to for 'non content' changes [Citation Needed]- this button gives the editor a subtle 'in context' reminder, that changing or adding this feed url is something worth checking, and it's convenient to do so at the point of entry)
But what about the Property Value Converter ?
Well the work in the plugin to pull back the feed contents based on the feed Url may as well be made available to be used in your razor view or surface controller...
so if your property has an alias of rssFeedUrl, something like this will now pull back the feed contents:
The FeedResult class has four properties, the FeedUrl (in case you want to do something with the raw url) and a boolean flag 'HasFeedResults' that you can use to determine if there are any feed items to display, if there is an error, it will be logged and the StatusMessage will give some indication of the message. Finally the SyndicationFeed property is the .Net SyndicationFeed class mentioned above, and contains all the details you need to know about the RSS Feed you are consuming and it's feed SyndicationItems.
It's worth putting the above in a Macro/CachedPartial/OutputCached Surface controller to add some caching so you are not requesting the feed ever time the page is refreshed.
Anyway, the snippet above produces something like the following:
but you can customise the html and css to use with your feed content.
... now the next time you are asked to add an RSS Feed to a page, I cannot promise you that your inward sigh won't still occur, a simple property editor can do nothing to temper the sigh of a developer that has been asked to add an RSS Feed to a page for seemingly little reason by very well meaning people, but I can hope, that the thought of demo'ing this small crumb of helpful functionality and the smile of gratefulness you may receive in return could temper you from responding with a tired sulky "really?, well it's technically possible but...
So recently we helped upgrade one of our support client's Umbraco sites from v6 to v7, all went very smoothly, except suddenly...
...puncturing their haze of post upgrade bliss was the shocking realisation that they were entirely in love with the Cogworks CogPageReview dashboard and actually their entire world of workflow revolved around this simple content review date dashboard - and it wasn't in fact 'something they didn't really use and could live without' - but something they needed superfast, or otherwise, well... some content might get old and nobody would know....
I've come across this deceptively simple package a few times before, and people just tend to believe 'it's part of Umbraco', particularly for large sites containing public information that will only be valid for a certain period of time, you set the content review date, and don't have to remember to check next year, it's invaluable.
Anyway onsite I knocked together quickly a dashboard to do a similar job in Umbraco 7, and you can download it here for the time being, if it helps you out, and hopefully we can push this back into the original package, but what I have here is pretty hasty, it's working so far... fingers crossed.
... but hey that isn't what this blogpost is about...
So what I wanted to blog about was three small things I noticed putting this together that I sort of didn't really know about or had learnt recently from the above course and also had spotted in the refactoring of the 'proof of concept' (for proof of concept, read not very good) Redirect Dashboard that HQ ahem refactored for the 7.5 release - that seemed new to me and I thought were mildly worth talking out loud about, here they are:
1) Don't use $scope 'willy nilly'...
So I'd got into the habit of using $scope all over the place, and when we introduced property editors to the Umbraco Level 2 course we taught this approach - officially from the docs:
$scope as a Data-Model "Scope is the glue between application controller and the view." and so enables your directives, controllers and views to share values and communicate in an agnostic manner.
Anyway cool kids don't seem to do it like that anymore - looking at the Redirects dashboard the preferred pattern moving forward for Data Models is to declare a ViewModel object or vm for short!
so here I'm grouping the dashboard status and settings on an object and for some reason to me it seems to make the controller much much easier to read...
... and you don't need to inject $scope into your controller unless you are using it for non data model purposes, eg setting watches and the like.
2) Backoffice styling - flexgrid and pagination, particularly pagination.
Making things look good in the backoffice has always been a bit of guesswork over what is there, and providing your own styles, which then as Umbraco changes start to look a bit weird... what package creators need is a style guide for property editors and dashboards that they can use to keep the look and feel of their creations consistent with the vanilla backoffice... the good news is I'm told the style guide is very much a planned thing that will happen soon.In the meantime if you are presenting tabular data, the redirects dashboard has the most recent HQ approach to this kind of thing and the first thing you notice is Flexbox for table layout is in the core, was that always there? I don't know, but it's certainly worth being aware of and utilising!
The most exciting thing though for me, because I hate implementing paging... is there is a new directive called 'umb-pagination' <umb-pagination - to consistently handle paging ui in the backoffice, and it's really easy to repurpose: so all I needed to do here was to create a pagination object on my viewmodel with totalPages, pageNumber, etc, populated from my search results
also we have an <umb-loading-indicator directive to handle the interaction when it takes a while to build the report
and an <umb-empty-state to show a message when there is no content review...
3) Finally language translations of text within the dashboard
I've always kind of wanted to provide the opportunity for people to translate the text used within a property editor or dashboard I'd created, but never really understood how. It's not just cos I'm English and think people from other countries will undertand if I just talk louder... anyway this isn't new, but is all setup by convention and so it might not be apparent what to do.
In your angularJS dashboard view or property editor you can use the <localize tag with a key and a default value (there is also a localize service if you need to work with the translated text in your controller)
you create a different file for each language you wish to support.
It's then easy for someone to copy your lang file, and create their own if they decide to use your dashboard or property editor with Spanish editors etc
In conclusion
It's well worth attending that 'Extending the Umbraco backoffice' course
.
