Moving Back to WordPress from Habari :(

Over the course of 7 days, I completed the migration of this site from Habari 0.7 to WordPress 3.1.2. I ultimately came to realize that I needed to spend more time writing my blog posts and less time fiddling with my site or theme. In the end, I needed a fully baked CMS as Habari was still a little too ‘rare’ but the Habari CMS will always be on my radar. I knew this sad day would come and I already talked not-so-seriously about it with my post about navigating with Habari. Here, I will talk about the little issues that made my decision to switch somewhat easier, what it took to move my site from Habari back to WordPress, automating the process, more caveats as well as as some interesting things I miss about Habari!

Why?

My case with Habari is probably skewed by the fact that previously,Jane Talks Tech! ran on WordPress. Then, I switched to Habari. At the time, I had several comments on the site, I wasn’t using the Disqus commenting system then and I don’t even think they had the feature to migrate your existing comments to the Disqus servers. I didn’t have the foresight to do as much testing as I did in this reverse switching scenario so I switched to Habari, I experienced several issues that were later corrected/forgotten about or unfixed. In any case, here are some of the issues with Habari I’ve experienced or was experiencing:

  1. Trackbacks/Pingbacks: With Habari, trackbacks/pingbacks behaved very unexpectedly. I would write a post and link to a different post I’d written; More often than not, the trackback/pingback would be truncated i.e. would be missing the closing [/a] tag for links which would then cause the comments box & any associated text to be hyperlinked! I eventually went through *all* my trackbacks and manually edited them to either add the missing [/a] tag or remove the hyperlink all together. This would come back to bite me as I discovered during my migration back to WordPress but I had no choice! I know there’s a difference between a trackback & pingback but for my purposes, all I cared about was that:
    • Jane publishes a post and links to 1 or 2 more posts written by her.
    • Jane may edit published post and through changing post title, may inadvertently change the slug for the post.
    • As Jane discovers during her migration back to WordPress, there are several broken links in pingbacks/trackbacks that could only arise as a result of post title/slug change.
  2. Post title/slug problems: Every webmaster or blog owner should have a Google Webmaster Tools account. You will discover important things about your site in Google’s eyes such as broken links, search queries, etc. I didn’t login to my Webmaster tools account as often as I should, but for my migration back to WordPress, I finally took a look at what was going on with my site. To my horror, I discovered dozens of broken links to several internally linked posts. Majority of these broken links came from the realization that at some point, I had changed the title of certain posts which (and I’m not clear about this point) presumably changed the slug which would alter the URLs for those posts. In fairness, Habari doesn’t claim to be all that and kitchen sink. Habari is blogging software (first & foremost) and any work to extend the functionality of Habari is via plugins. However, I can see now that I need assistance in form of plugins and that since I cannot code a plugin, I need to go somewhere where plugins are already available. I cannot explain why I had so many bad/truncated URLs but I do know that the trackback/pingback issue with truncated URLs contributed. Whose fault (mine or Habari’s) that is doesn’t matter anymore. 🙂 I encourage anyone who experienced this same problem to leave a comment. The moral lesson of this bullet point is:

    Be careful in changing the titles and inadvertently changing the slug for already published posts because your pingbacks/trackbacks may not be updated.

  3. Threaded comments: When I switched from WordPress to Habari, I lost the threaded comments I had but I wasn’t too broken up about it. I kept the Habari commenting system and after a couple of months, I enabled Disqus commenting on my site. Unlike the Disqus plugin for WordPress, there was no feature in the Habari Disqus plugin to migrate existing comments to the Disqus system so comment management for my site was a little split between checking Disqus.com for moderated comments and checking on my Habari comment management page (because on posts containing existing WordPress/Habari comments, the Disqus commenting box did not appear which was intended behavior). I got over this but I sorely missed having the fully fledged Disqus plugin for WordPress. This would NOT be an issue for someone starting a brand new blog without comments. For someone with an established blog & comments, I would definitely recommend LOTS of testing before migration.
  4. Theming/Themes: With Habari 0.6, there were 2 gorgeous themes I’d found and absolutely loved. They were Dark Autumn by Ali Dmondark and Georgia by Thomas Silkjær. More importantly, they displayed fine without me needing to dive too deeply into CSS. With Habari 0.7’s release, it was trial & error to find out which themes worked OOB (out-of-box). There is a distribution hub for Habari themes but it is a work in progress. Habari is a labor of love and I can empathize with plugin or theme developers who do not have time to work on updating their themes/plugins with either bug or feature requests. Even Dark Autumn and Georgia weren’t without their quirks, but I was able to bend those themes to my will. 😛 With 0.7, those themes didn’t work for me (Dark Autumn was updated but I still had niggling issues I couldn’t fix and life got in the way) and with everything else going on, I figuratively threw up my hands and left the default K2 theme. I suspect the lack of themes is probably hurting the adoption of Habari. It probably doesn’t help that the only *definitive* post on creating themes with Habari is over 2 years old (it actually refers to porting WordPress themes) and that needs to change. There was actually a thread on Habari’s user mailing list where a Habari user was quite confused by reading the Habari Wiki page for creating custom Habari themes! After twittering about my switch back to WordPress, a Habari dev reached out to me and I told him one of the major reasons for leaving was the themes. [blackbirdpie url=”https://twitter.com/#!/janetalkstech/status/71723482823659520″]
  5. Habari Plugins: I like to compare my switch back to WordPress from Habari to my switch to Android from Symbian. Is Habari powerful with cleaner code, etc? Yes. However, the theme & plugin ecosystem for WordPress is huge which makes it the default release for several newfangled web services on the market and reminiscent of how apps are typically released on the iPhone first before Android. Nevertheless, there *are* plugins for the major sites for Habari and as a testament to the dedication of the Habari developers, a good number of these working plugins were written by them. For instance,
    1. Photosharing: There are plugins are available for viewing/inserting photos from: Flickr (bundled by default with Habari), Smugmug written by Colin Seymour, Picasa silo written by web development firm, Second Variety, Photozou, etc
    2. Social media: There are plugins available for viewing/inserting tweets, sharing blog posts, etc.
    3. Code sharing: There are plugins for embedding source code in blog posts, and many other plugins on the Habari distribution hub.

    However, WordPress wins out when it comes to the amount of customization that is possible with the help of plugins and I have truly come to appreciate the more mature plugins for services like Google Analytics, SEO, etc which keep me in my dashboard and help me continue to write.
    [blackbirdpie url=”https://twitter.com/#!/janetalkstech/status/72295045386076160″]

How?

Now, let me dive into the really cool details of how I managed my switch from Habari to WordPress without losing my sanity or (hopefully) not ruining my site’s ranking in Google’s eyes! Over a year ago, Chris Meller (a Habari developer) whipped up a small script to aid anyone who wanted to move back from whence they came i.e. WordPress. He used a beta version of WordPress 2.9 but WordPress is now at version 3.12! Currently, Chris Meller’s WordPress migration script will move the following:

  1. Posts – published or draft
  2. Pages – published or draft
  3. Comments – approved only
  4. Tags (Note: The script didn’t import my tags for some reason. This is an important warning that, depending on your setup, could be time-consuming work. I talk about how I got around this so keep reading. :D)

So, here are the first steps I followed to get my posts, pages, & comments imported into the test WordPress 2.9 installation and getting to a base installation:

  1. Create a new directory on janetalkstech.com.
  2. Downloaded a copy of WordPress 2.9 from the WordPress release archives as I saw no reason to tempt the gods by attempting the migration with a version of WordPress *he* didn’t use.
  3. Installed WordPress; This version will use the default “admin” username so I made sure I created the same username I had with my Habari installation and made that username an “Administrator”.
  4. Updated the WP Migration script with the database details for the new WordPress installation per Chris’s instructions:

    Well, you dump it in your Habari root directory, edit the array at the top so it can connect to the MySQL database your WordPress instance is installed in, and you load it up in your browser. You should see a bunch of junk about things it finds – and hopefully no MySQL errors along the way.

  5. Uploaded the edited WP migration script to my Habari installation at janetalkstech.com and *hopefully* no major errors in your case. In my case, a ‘major’ problem was the fact that my tags weren’t imported but I was just euphoric that I didn’t run into any “showstopping” bugs.
  6. Upgraded the test site to WordPress 3.1.2, blocked search engines from accessing the test site to prevent duplicate content problems, turned off commenting/pingbacks/trackbacks because this was a test site and installed tracking code (Google Analytics & StatCounter) to keep tabs on the visitors to the special test install site.
  7. Installed the WordPress DB Backup Plugin and User Role Editor plugin. These two plugins will save you heartache. Just do it now and thank me later.
  8. When I started the move, I had exactly 1, 111 tags in my Habari database and over 150 posts. The WordPress migration script did not import my tags and accordingly could not associate the posts with the tags. This was depressing because I was looking at *hours* of copying & pasting and doing tedious data entry. So, what did I do?

OutSource The Tedium

  1. Elance: Before all of this began, I initially created a job on Elance asking for a developer to take care of the mess for me. I got bids ranging from $200 to $300 from non-US based workers. As I was able to get my posts/comments/pages imported, I then created a new job asking for data entry work to add tags & associate the posts with corresponding tags. I got bids for this job ranging from $50 to $250. I wasn’t happy with the workers that were bidding on my jobs so I eventually canceled my jobs and decided to give Amazon’s Mechanical Turk program a try.
  2. Amazon MTurk: The website to sign in as a “Requester” is https://requester.mturk.com. The website is rather confusing so I recommend you skip straight to “create a hit individually“.
    1. You will be asked to pre-pay for your HITs so make sure you have your credit card info on hand or on your Amazon account.
    2. Err on the side of too much information. I created 2 jobs: one was asking for the creation of my 1, 111 tags based on the live site at Jane Talks Tech! for $10. I created screenshots showing how to create a tag, properly create the slug and submitting the new tag. The other *very* important thing I did was install the User Role editor plugin for WordPress. I didn’t have any extremely sensitive information in my drafts or blog posts so I wasn’t too worried about granting someone access to my test site. However, the User Role Editor allowed me to make sure the new user was was only able to create tags! Once the tags were created properly, the worker was paid. This worker was able to complete the job in less than 3 hours so I wasn’t left in limbo.
    3. The other job was for updating the posts with the created tags based on the archives at Jane Talks Tech! After Worker #1 was done, the next Worker to accept my HIT was to work on adding the tags to each post on the test site. Again, I created screenshots showing exactly how to do this and the worker was able to complete this task in less than 24 hours!In summary, for less than $25, I was able to get the 1, 111 tags created and added to the correct posts via Amazon’s Mechanical Turk program.
    4. I also ended up removing ~ 150 tags that were incorrectly added but thanks to the excellent “search tags” option in WordPress, I was able to find a common phrase within the bad tags and bulk-delete them. 😀
  3. One important thing I did (& kept doing) after major steps like:
    1. running the WP migration script successfully
    2. upgrading to WordPress 3.1.2
    3. creating tags thanks to an Amazon Turk worker
    4. updating posts with correct tags thanks to an Amazon Turk worker
    5. removing invalid tags from posts (done myself) was:

    WordPress Database Backup Optionsbacking up my WordPress database files via PHPMyAdmin where I downloaded the entire database and via the WordPress backup plugin three different ways: email, server and download. It is also a good idea to export your content using the built-in WordPress exporter which outputs a WordPress eXtended RSS file. The idea being: if something goes wrong, you can always fire up another WordPress instance and get going without having to start from a blank/fresh WordPress install.

Cleaning Up & Watching Out!

This part cannot be outsourced, in my opinion. So plan on having 2 – 4 hours to spend on fixing and getting things just right. My major concerns once I had my posts, pages, tags and categories set up properly were making sure: embedded/linked image paths were updated, internal/external urls were updated & functional, updating my sitemap and migrating my comments to the Disqus commenting system.

  1. Updating Image Paths: WordPress’s standard URL for uploaded files is: http://sitename.com/wp-content/uploads/whatever while Habari’s standard URL for uploaded files is: http://sitename.com/user/whatever/. So, I:
    1. Copied over my pictures & screenshots from /user/files to one folder in /wp-content/uploads/ and installed the Search Regex plugin for WordPress. I then performed a simple search for the “/user/files” string and replaced it with my WordPress folder path. Again, please backup your database files several ways before running this plugin. It is very powerful and done wrongly, can cause serious problems. So before running this, backup; after running this, backup. 🙂
    2. Installed the Broken Link Checker plugin for WordPress which alerted me to more broken image paths and I was able to use the Search Regex plugin to replace the links if there was a pattern. As I’ve become accustomed to, I backed up my database files before and after running this plugin.
  2. Update Links: The Broken Link Checker plugin was invaluable during this process particularly because it let me update the link without manually editing each post! This link checker works on all internal and external links so make sure you don’t constantly ask the program to crawl your links. There are more ways to check your site for dead/bad links but for immediate results or a snapshot of how bad things are, you cannot beat this simple plugin on small sites like mine. Another useful feature of the Broken Link Checker plugin is that it also detects redirected links. That was how I discovered over 200 internal links which had a slash at the end. This is probably a WordPress-related matter, but with the Broken Link Checker plugin, I was able to update the links with the canonical urls and avoid the extra slash.

    Thanks to this plugin, I was able to weed out several dead external link and discover several bad pingback/trackbacks that were hold-overs from my Habari days of broken trackbacks/pingbacks.

  3. Updating Your Sitemap: With the help of Yoast‘s WordPress SEO plugin, I was able to make sure that my site was verified with Google Webmaster tools, Bing Webmaster tools and Yahoo Site Explorer. I was also able to regenerate a sitemap of my site and resubmit that to the search engines. The WordPress SEO is a very polished plugin that does more than generating a sitemap; It helps you optimize your site and each post or pages for select phrases/keywords!
  4. Switching Comment Systems: This was probably the easiest part but nonetheless still nerve-wracking for me. As I alluded to, the official Disqus plugin for WordPress automagically imports your comments into your Disqus profile. I absolutely love Disqus because it allows you to get a fuller picture of your commenters and they’ve proven themselves honest by allowing you to export your Disqus comments back to your WordPress database or exporting all your comments to an XML file (which I am reasonably certain will be supported by whatever other CMS usurps WordPress’s throne).
    WordPress comments queued for Import into DisqusDisqus Successfully Imports WordPress comments

Missing Habari Already

  1. Media Management: Habari has a unique method of managing uploaded media (like photos, documents, etc). In the developers’ words:

    To assist in the management of media, such as images, video and audio, Habari provides a mechanism for defining virtual filesystems for media, called Media Silos. A silo provides a consistent interface for dealing with media.

    This made browsing through my Flickr/Smugmug albums an absolute pleasure and I still have not found a comparable Flickr/Media managing plugin for WordPress. 🙁

  2. Speed/Errors/Timeouts: The Admin backend for Habari loads drastically faster than WordPress’s. Oh, and don’t get me started on the errors I get even when trying to view my Admin page in WordPress!

    I’ve already had more 404 errors in the course of saving or doing anything in the WordPress backend this week than I have in the year of using Habari.

    Crazy, but again, I am aware that I’ve made some stability trade-offs in moving to WordPress because I want in to the WordPress plugin/theme ecosystem. 🙂 In fairness, WordPress is now a fullblown CMS whereas Habari can be called a minimalist blogging system.

The TL;DR version of this post is:

  1. Jane Talks Tech! now runs on WordPress. 😛

Overall, I’m very pleased that my migration from Habari to WordPress went smoother than I imagined. I didn’t have to visit IRC once or send flares into the digital universe. 🙂 If you have any questions/comments, drop  a note & I’ll do my best to answer.

Navigating Habari, the content management system – Part 1

It’s been over eight months since I migrated my blog (Jane Talks Tech!) from WordPress to Habari. It wasn’t done lightly and I did my due diligence by creating a test installation of Habari, making sure the importer worked, ironing out issues (credit entirely due to the devs in the Habari IRC channel) and finally pulling the trigger. That said, my experience with Habari has been a rollercoaster, but I’m still hanging on. 😛 I feel connected to this fledgling blogging system and believe it has a strong chance of surviving in the crowded content management space. Here are my thoughts on things that could make Habari even more of a contender today and ways you can help make it happen. So here goes:

There are a lot of Habari plugins listed in the official Habari repository. However, beating a dead horse, a small amount don’t work for the stable release version of Habari (i.e. v 0.6.4). Those that work are great at what they do and serves to tease me about the possibilities with Habari! I’m currently working on my bare-minimum and recommended list of Habari plugins. The list below picks on select plugins that would be pretty cool to have working in Habari v. 0.6.4:

  1. Polls: I found out about this plugin for creating polls, but it works with the bleeding edge or trunk version of Habari. According to a tweet by one of the Habari developers, version 0.7 of Habari might be hitting stores sometime in August. So until then, I’m hoping the community can help to polish some of the plugins.
  2. Video Embeds: Currently, besides grabbing the embed code from whatever video-sharing site you are on, there’s no relatively easy (as compared to WordPress) way to share videos. I’m thinking of a plugin that when activated, will allow you to plug in the unique ID number of the video and have it be recognized, etc. Not a biggie, but it removes yet another hurdle to the content-creation aspect of things. 🙂
  3. Photo Sharing support: When I started with Habari, I was a SmugMug user and I made good use of the fantastic SmugMug (#shamelessplug: coupon DGwxx0rcYaTWQ) silo plugin by Colin. Then, I switched to Zenfolio and wouldn’t you know it, no plugin for Zenfolio (#shamelessplug: Referral Code: WRC-655-RUY). Again, Zenfolio has code that can be embedded into blog posts as well as direct links, but man, I’d love to have a Zenfolio silo for Habari. 🙂 There are several photosharing sites and it’d be unfair to ask that the developers create site-specific plugins for the dozens of sites out there! This is where the community steps in and I’m hoping the next Habari developer on board has a Zenfolio website.
  4. Plugin/Theme discovery and updates: The official place to get plugins & themes from Habari is http://habariproject.org/dist/themes and http://habariproject.org/dist/plugins. One thing that would be nice is for there to be demo wherein people would preview themes without downloading, uploading and activating on a live blog or for the more technically inclined, a test blog. With the Habari plugin, I’d like to see some way to sort the plugins. Off the top of my head, perhaps it would be good to have folders corresponding to 0.x habari releases and then having *only* 0.x compatible plugins loaded in that directory. Plugin update notification is another pain, but so far, I’ve simply just check the plugin distribution site to see if there are changes. Perhaps an email subscription or a way to monitor changes to plugins would be a way of notifying users that plugins have been updated. Documentation with regards to plugin installation is an area that sorely needs help. A select few of the activated plugins on my blog use the neat ‘Help’ icon and some plugins lack readme files. Not a big deal because I’m a big fan of users educating themselves, doing due diligence by reading the manuals (in the case, the plugin.php files can be quite helpful) and asking for help before giving up. 🙂
    Habari Help icon

    One way I’ve been ‘trying’ to help is letting the devs know my problem(s) via Trac and writing on my blog about problems I’ve face and my workarounds or tweaks. Any CMS user will understand that with plugins and themes, there’s the almost-obligatory post-activation code and theme-tweaking (gravatar styling for comments, adding edit links to posts, adding code to certain parts of the themes as part of plugin installation, etc). Fact of life, but clearly documenting what needs to go where will go a long in easing the pain.

That said, I always find new Habari-powered blogs daily and I also check the Habari Plugins repository frequently. You can get an alternate listing of Available Habari Plugins and Available Habari Themes on the Habari Wiki. I also just found about an unofficial community theme repository called the Ziada Project which pales in comparison to the official repository so far.

Minor gripe: I’ve so many different Habari themes in the wild and speaking as a complete user & parasite, I selfishly wish that those coders who’ve created new/interesting themes should please consider releasing some themes. I’ll leave your link(s) in the footer/sidebar/wherever. 😛 Heck if I can’t get excellent themes for free, I won’t be opposed to paying for one! As I tweeted earlier, I can’t wait for the rise of premium themes in Habari. 🙂

So far, I’m still hooked on the Habari content management system for technical reasons I know nothing about and the fact that Habari comes with less cruft. The Habari cabal is aware of most of the shortcomings and from being a lurker on the habari-dev mailing list, they’re listening to feedback. With the demands of IRL work and personal stuff, they’ll get it done. 🙂 In fact, Trac is where much of the behind-the-scenes magic keeps happening as you’ll see stuff being fixed/updated.

For new users of Habari, if you run into trouble of *any* kind, the absolute best thing you can do is visit the Habari IRC channel.

Here’s how:

  1. Download and install an IRC client like mIRC or XChat or TurboIRC or .. My point is: there are several free options for getting an IRC client up & running.
  2. Once you have installed your IRC client, connect to the Freenode network which should be one of several options on whatever Chat/IRC client you have. Or you can click this link: irc://chat.freenode.net/ and it should allow you to open the Freenode network in your default IRC client.
  3. Once you have connected to the Freenode IRC network, connect to the #Habari channel by typing #Habari when prompted to connect to a channel.
  4. Once in the Habari IRC channel, ask your question and make sure you are able to make on-the-fly changes to your blog and provide feedback to the developers.
  5. OR, did you know that there’s a Live Help for Habari plugin? Yes, you can pretty much forget everything I wrote in the first 4 bullet points and simply install the LiveHelp plugin. It connects you directly to the Habari IRC channel from the comfort of your Habari blog administration panel! I love it and it’s frankly easier to do that installing and all the fun stuff. 😛
  6. OR, if you don’t care to be logged into your blog’s administrative interface to use IRC, simply visit http://webchat.freenode.net/ and join the channel #habari.

For users who have issues that could be considered bugs or if not fixable via IRC, you can always file a ticket in the Habari Trac system.

Here are some basic tips about using the Trac system:

  1. For issues that are NOT because of a plugin or theme being broken or incompatible, visit https://trac.habariproject.org/habari and create an account. It is extremely helpful if you can copy & paste the error codes you’ve seen into a text file and then upload that file as part of your bug filing. I’m also a fan of taking screenshots of the relevant details and uploading those as well. The point is that any helpful information that will lead to the solving of that bug is highly appreciated.
  2. For issues that ARE because of a plugin or theme being broken on incompatible, visit https://trac.habariproject.org/habari-extras and as you can see, I am rather whiny. 🙂
  3. Whatever you do, be honest about the severity i.e. if it’s a wishlist for a plugin or an ‘extra’, don’t mark it as a showstopping bug or as major or urgent. Remember the golden rule. 🙂

For users happily coexisting with Habari and want to help out:

  1. Spread the word about Habari via tweets, articles, blog posts and posts on your social networks. I haven’t been too shy about contacting people as asking/letting them know about the new kid on the block. 🙂
  2. Contribute to Habari via creating plugins or themes for Habari and/or contributing to the codebase or helping with documentation.
  3. Notify the developers of security issues or file tickets about bugs.
  4. Join a Habari working group to dedicate yourself to any issues you’re passionate about.
  5. Join the Habari forums, Habari Users mailing list and/or the Habari Developers mailing list to be of assistance to newbies like me and developers new to Habari.

Either way, I’m eager for v. 0.7 of Habari.

It is thisclose to release and I’m excited to be a part of this burgeoning community even if my most significant contribution was a plugin request. 😀 For a list of blogs running Habari, visit http://wiki.habariproject.org/en/Habari_Powered_Sites. I’m aware that because Habari’s still growing, it might not be for everyone. Here’s a link that describes how to migrate your content back to WordPress if that’s where you came from. Earlier this week, I did a test migration back to WordPress 3.0 with this same script and it worked. My posts, comments, user(s) and tags and pages were properly imported into the new WordPress installation, but my categories were gone i.e. all posts were listed as Uncategorized. I don’t think this is a dealbreaker. I’d be more worried if my tags weren’t imported because tags are another way of keeping things ‘categorized’ if you will. Just a FYI. 🙂

Tip for attempting a test migration from Habari to WordPress: use a new WordPress install with a new MySQL database and new username. It’s ridiculously easy to do on Dreamhost so you have no excuse not to. I just like things clearcut and that’s my only reasoning. 🙂