I was looking forward to wrapping this extension up within the first couple days of xmas break (my workplace shuts its doors between Christmas and New Year’s), but then every single one of my kids got sick in turn. All of them. It was a crappy week.
But now it’s finally done (I also wrote about it here and here), and I can generate bookmarks from Pinboard tags the way I wanted. All in all, a good learning experience.
If you want to check it out, it’s in the Chrome store.
Everything after this point is just random observations and stuff I learned in the process.
Contents
- 1 Recursion
- 2 Tracking Issues
- 3 Customizing jsTree
- 4 Reinventing the Wheel… or not
- 5 Storage in Chrome
- 6 Catching Chrome API Errors
- 7 Test
- 8 The Last Mile
Recursion
When a user selects Pinboard tags to generate bookmarks from, they’re added to a treeview where they can further arrange and organize them, creating a structure that might look like this:
Traversing the selected tags in the treeview, which could be nested to any depth, was a prime candidate for recursion. I like recursion. It’s a mental exercise, trying to figure how to call a function recursively without screwing everything up.
The important thing about recursion is, if you forget to terminate it it’ll go on forever! Every recursive algorithm needs a terminating condition. Something that tells it when to stop so you don’t end up with an infinite loop (and possibly a stack overflow exception).
In the code snippet below, I detect when a folder is about to be created (the “type” for a folder is “default” in the treeview), and then pass the folder’s children to itself, to recursively generate bookmarks as it traverses deeper and deeper into the nested list of children. When all the children of the deepest-level subfolder on a given branch have been processed, the stack will unwind, traverse down into other subfolders, and eventually back all the way out and finish. For every folder, createPageOrFolder will be called 0 or more times.
function createPageOrFolder(parentNodeId, tagNode, urls, ignoreDelimiters, storeRootId = false) {
tagNode.forEach(function(tag) {
if (tag['type'] == 'default') {
chrome.bookmarks.create({'parentId': parentNodeId,
'title': tag['text']},
function(newFolder) {
if (storeRootId) {
// store ID to be used for deletion later
}
createPageOrFolder(newFolder.id, tag['children'], urls, ignoreDelimiters);
});
} else {
urls.forEach(function(url) {
// create individual bookmarks
});
}
});
enableInputElements();
}
Hmm, while writing this in here I realized I have a small bug in the above code.
I disable most of the GUI when a button is pressed. That last line above that calls enableInputElements() will re-enable the controls. That’s all fine and good, but since the method is recursive, it’s eventually going to unwind when it’s done processing all the tag nodes. And at that point it’s going to finish running all of those calls to createPageOrFolder (one for each folder or tag in the treeview), and execute enableInputElements an equal number of times.
It didn’t result in screen flicker or noticeably longer execution time, but it’s unnecessary. I moved that line of code into the method that called createPageOrFolder the first time.
Tracking Issues
GitHub supplies a few ways to track your work. I had created a couple dozen issues for each of the (sometimes tiny) tasks I was trying to accomplish in this extension. It seemed heavy-handed at times. I like kanban boards like Trello for rapidly creating, organizing and destroying small tasks.
As it turns out, GitHub has a “Projects” feature, which is basically a kanban board with cards you can quickly create and drag around. It’s not as intuitive or feature-rich as Trello, and the limited viewing area and character limit is annoying, but it’s one more tool to consider when you’re managing a project in GitHub. Here’s part of the one I used for what I called “release 1.0”.
Customizing jsTree
Last time I wrote about this, I had updated the jsTree.js file directly which felt wrong. It’s like modifying an established library you downloaded instead of extending it. Or modifying a theme in WordPress directly, instead of creating a child theme. It’s usually better to leave the original in place, and override the parts you’re interested in customizing.
I eventually figured out the correct way to customize jsTree. There’s not much to say about it, but here’s a portion of that code which might be interesting to someone else trying to do the same. You can set the node types, enable plugins, modify the sort algorithm* (here I’m sorting by “type” so folders would appear first, whereas the default sort only takes “name” into consideration)*, etc.
$('#tagTree').jstree({
'core' : {
'animation' : 100,
'themes' : { 'stripes' : false },
'multiple' : false,
'check_callback' : true,
'data' : data,
},
'types' : {
'#' : {
'max_children' : 1,
'valid_children' : ['root']
},
'root' : {
'icon' : '../../images/tree_icon.png',
'valid_children' : ['default']
},
'default' : {
'valid_children' : ['default', 'file']
},
'file' : {
'icon' : '../../images/bookmark_icon.png',
'valid_children' : []
}
},
'plugins' : [
'contextmenu', 'dnd', 'sort',
'state', 'types', 'wholerow'
],
'sort' : function (a, b) {
return this.get_type(a) === this.get_type(b)
? (this.get_text(a).toLowerCase() > this.get_text(b).toLowerCase() ? 1 : -1)
: (this.get_type(a) >= this.get_type(b) ? 1 : -1)
},
'contextmenu' : {
'items' : function (node, callback) {
return {
"create_folder" : {
"separator_before" : false,
"separator_after" : false,
"_disabled" : node.type == 'file',
"label" : "Add Folder",
"action" : function (data) {
var inst = $.jstree.reference(data.reference);
inst.create_node(node, { "text" : "New Folder" }, "last", function (new_node) {
setTimeout(function () { inst.edit(new_node); },0);
});
}
},
...
// lots more context menu stuff
...
};
}
}
});
Reinventing the Wheel… or not
I needed to encode some of the values I coming from Pinboard, but when I searched for how (figuring there would be something built-in to javascript to handle it), I kept coming upon fixes like this one…
var elem = document.createElement('textarea');
elem.innerHTML = encoded;
var decoded = elem.value;
Seriously, am I the only one that smells that?
By and large, the accepted solution seems to be to create an HTML element (like here and here) solely for the side-effect that it happens to encode certain characters. Stuff like that should be a huge red flag, and I can’t believe they’ve typically got so many upvotes.
Instead, I found a library called HTML entities (HE) that does just what I need, and handles a lot of cases that the above hack most likely doesn’t.
If you ever use it, and you happen across an error like this one, make sure you’re using the “he.js” file from the root of the repository and *not *from the “src” directory. That one is just a template with a bunch of place-holders to be filled in later, and it won’t run as-is.
Uncaught SyntaxError: Unexpected token <
The one in src contains placeholders for complex data that is generated and placed into the code at build time.
Storage in Chrome
I’m really struggling with storage. There are two types of storage in Chrome. One is “local” storage, and it’s a 5MB bucket per extension with no limitations. The other is “sync” storage used with the “sync” feature in Chrome. I wanted to sync selected tags and the other options but ultimately decided not to.
The sync storage is too restrictive – a 100KB bucket and all kinds of limitations on how and when data can be stored in it. It’s pretty obvious they’ve intentionally made it difficult to sync anything more than small bite (byte?) sized pieces of data, placing the onus of sync’ing on each plugin provider. Well, I don’t have anywhere to sync this stuff for anyone, although looking into allowing users to use dropbox or the like might be interesting.
Sync comes with too many considerations anyway. For example, I mentioned earlier that I store off the IDs of the top-level folders and bookmarks I generate so they can be deleted later. Those IDs are guaranteed to be unique in a profile, but it doesn’t mention whether they remain unique when Chrome’s own sync’ing mechanism recreates the bookmarks on another machine. If not, then my code that deletes previously generated folders and bookmarks may unintentionally delete the wrong ones on another machine.
Even though the local storage has no restrictions besides size, it doesn’t quite work for everything I want. Right now I have 3277 bookmarks that total 858.4 KB in storage, which comes to roughly 268.25 bytes per bookmark and means local storage can handle roughly 19,500 bookmarks. That’s a large amount but by no means inconceivable.
Checking out the Pinboard stats, it looks like under 6000 bookmarks may be average, or twice the number of my own, but that could leave some outliers with tens of thousands. It was important to me to be able to store those bookmarks locally since Pinboard limits requests to get all bookmarks to once per 5 minutes, so I took other measures and did the best I could without using storage for that particular purpose.
Catching Chrome API Errors
I have all kinds of logic spread throughout my code to catch and log (and potentially display to the user) various errors. But some exceptions seemed to keep getting through.
After some reading I discovered that many of the Chrome API calls, on encountering an error, set a chrome.runtime.lastError variable. So that was something unexpected. Seems like it’d make more sense to just throw an exception like everything else. Here I’m checking that variable and then, if it’s defined, logging the message it contains.
$("#generateBookmarks").on('click', function() {
disableInputElements("Generating Bookmarks... this may take a minute");
chrome.storage.local.set({'selected_tags': $('#tagTree').jstree(true).get_json('#')}, function() {
if (chrome.runtime.lastError) {
logError("Unable to save your selected tags to storage:\n\n" + chrome.runtime.lastError.message, false);
}
getAllPostsAndGenerateBookmarks();
});
});
$("#saveTags").on('click', function() {
chrome.storage.local.set({'selected_tags': $('#tagTree').jstree(true).get_json('#')}, function() {
if (chrome.runtime.lastError) {
logError("Unable to save your selected tags to storage:\n\n" + chrome.runtime.lastError.message);
}
});
});
Also, I used console.log a lot while debugging, but if you ever find yourself in the same situation and you’re trying to log more complex objects, try console.dir. It provides a different output that’s easier to read in some cases.
Test
I should probably figure out how to write some automated tests at some point, especially if I learn more javascript. In the meantime, I wrote up a bunch of manual test scenarios I can run through to make sure I haven’t broken anything. Tedious, but it works for now.
The Last Mile
Like pretty much every project I’ve ever worked on that lasted more than an afternoon though, the closer I got to the end the more I just wanted to be done. The last mile of any race is always the toughest psychologically. Ooo, that’s quotable. Eh, I probably stole it from someone else.
Anyway, I feel good that I pushed through and published it. Maybe no one will ever use it, but I feel like I accomplished something by putting it out there.
One last thought… this totally sums up my feeling on CSS. Seriously infuriating. Ugh.
