MongoDB

Debug MongoDB Map Reduce with the Mongo MapReduce Web browser

I am in the process of rewriting this, if you want to see what I am excited about then checkout the repo here: https://github.com/angelozerr/mongo-mapreduce-webbrowser

You may, or may not, remember (depending whether you are new or returning to this blog) I wrote a post sometime ago about a new Map Reduce debug tool within your very own browser – using jQuery. My original sentiment on the subject was that it was still too rough to test, however, the author @angelozerr and @pascalleclercq have worked hard on the release of v1.0.0 with the intention of making it stable and boy, have they made it stable.

Here is the live demo. I recommend you try it out, I am going to blabber on down here about some of the things you can with it now.

First off, you are greeted by an interface with the folders containing your map reduces on one side and the potential preview of your files on the other. If you double click (I tried single clicking at first too) on any of the example map reduces then you will see it load up in the right hand pane.

Example Mongo MapReduce WebBrowser

Feel free to click on the image to zoom in, it should be 1080 compatible. From the screen shot you can see what I mean about the explanation. As an added note there is also a tabbed interface for map reduces (look to the top grey bar) for being able to load multiple files into memory at once.

Some of the key changes to the previous version I looked at is the editor (the right pane in the screen shot). It boats:

  • Code Completion
  • Syntax Highlighting
  • Code validation (with hint popups for telling you the exact error)

For each step of the map reduce. It will even underline errors for you so you can quickly and easily get to them without having to check what the icon in the side bar says. Too add a cherry on top, it also has a live preview where your edits will change the output immediately. All in all this is one slick application.

I had none of the previous problems I experienced, quickly and easily building my own map reduce and seeing its output.

There is still one problem that might catch people out: the input documents cannot take BSON objects such as the ObjectId() yet as such this means you cannot yet input:

[{'_id':ObjectId("455556676678"), "name":"sammaye"}]

That being said you can input documents in their deserialised format like:

[{"_id":{"$oid":"50d8fdedadd222278ba9090f"},"date":{"$date":"2012-12-25T01:14:21.312Z"},"url":"http;//iiiiiiiiiii"}]

However, it currently cannot convert console copied document down to this format. @ angelozerr has some ideas on how he wants to solve this. I am certain this is a feature that will soon be added in one form or another. If it had this feature it looks almost ready to use, I even tested this with around 1,000 documents on the count_tags.js example and it worked, it almost killed Firefox for a while but it worked and it did its job eventually.

I really do recommend now taking a look at mongo-mapreduce-webbrowser and trying it for yourself. It is definitely a tool that no MongoDB user (or map reduce) user should go without.

Advertisements

10 thoughts on “Debug MongoDB Map Reduce with the Mongo MapReduce Web browser

  1. Hi Sammayes,

    Many thank’s for your article. I have just started this project and you blog it yet. Wow , that’s cool:) As you have written, there are a lot of (little) problems to resolve. This project is not finished and you will have new features soon like syntax error, completion.

    Here some answer of your comments :

    1) document should be done once the development is finished.
    2) I have tested with FF and Chrome which have a great debugger. I don’t want loose my time with IE even if I test it quicly with it.
    3) you need a WEB Server because the webapp uses Ajax to display js files coming from a Folder and load dynamic scripts (when you wish to debug your MapReduce, you should before save (it saves the script and reload it) it to debug it). I use Embedding Jetty as WEB Server in my local machine, but in the live demo, Tomcat is used. You could implement the server with Node.js or other.
    4) there are a lot of debug, because I discover jQuery and it seems that jQuery 1.9 has removed several methods (like browser). The dynatree that I use works with jQuery 1.8, so I use jQuery migrate to support dynatree.
    5) to debug your MapReduce with Chrome for instance you open the Source scripts tab of Chrome, you do “Ctrl+0” and you type your MapReduce name to open the script and set a breakpoint. If you change the script in the textarea of the MapReduce, it’s important to save the script, close the Chrome Developper (F12), and reopen it (F12) in order to Chrome refresh the script in the Source panel.

    > My end verdict is that this program has undeniable potential but at the moment there is just too many teething problems to really test any Map Reduces correctly.

    I think the problem is the documentation too, but before doing that, I would like finish my development.

    Don’t hesitate to create issue with your problem at https://github.com/angelozerr/mongo-mapreduce-webbrowser/issues

    Regards Angelo

    1. Yes don’t waste your time with IE, if they ever even get a debugger that works properly.

      Awesome yeah, though if the documentation just added how to set up maybe a test envo for non-java people it could be really awesome for those who wanna break it :P. I do understand you wanna finish your development so I will wait in anticipation, I think this will be a good tool.

      Yea JQuery has removed a lot of functionality which they saw as useless, browser being one, since it was so unreliable.

      Ah ok; about point 5, does that go for input documents too? I wanted to test some of the examples with real world test set sizes to test the logic moves within the functions correctly, if I save the MR and reload firebug will it refresh the input?

      I think I will wait until initial development is done until I file issues :), I want to make sure you can focus on the important stuff to get this out.

      Overall, despite being new, it does have the a demo that shows its potential to be awesome so I am really looking forward to it.

    2. Yea requiring a server isn’t the end of the world and I have seen that actually you wanna add database handling to this program, so it seems like a good direction.

      Edited my article around it 🙂

      1. > just added how to set up maybe a test envo for non-java people
        I think the distribution should provide 2 products :

        1) a war to deploy teh webapp on any Java WEB Server.
        2) a jar + bin/sh to start the webapp with embedding jetty (no need to install somme Java Server)

        Once the webapp is stable, I think it should be cool if there are other server like Node.js. If you are interested, you are welcome!

        When I started this project, I had not a server, but it was very simply (you could jiust execute your (one) MapReduce but don’t save it and no debug). I have studied several means to manage files with WebBrowser (like File API, but you need to start your WebBrowser with some parameter to give write access), but server mode was the most powerfull.

        My idea is too share MapReduce between people. In a forum, you post “I have a problem with MapReduce, please see my MapReduce and you post a link to the online webapp, like http://jsfiddle.net/ for Javascript).

        Do you think None server mode is important?

    3. I would definitely be interested in making a PHP plugin for it (that’s my main language atm).

      None server mode isn’t important if those are your plans. A lite version later on when the full version is up can be made for that.

      A JSFiddle edition would be a killer and not just for MongoDB, people could make plugins for stuff like Hadoop and that.

      Yea, this definitely sounds good :).

      1. > A JSFiddle edition would be a killer and not just for MongoDB, people could make plugins for stuff like Hadoop and that.
        Cool, I’m happy my idea please you.

        To manage that, DB should be managed to persist the MapReduce on the DB (and not on the FileSystem like today).

        > A lite version later on when the full version is up can be made for that.
        Yes sI thought that too. For the moment I decide to manage only server mode.

        PHP plugin should be very cool, but I think you should wait that code become stable because the server side should change a little, but if you want you can start something and I will give you right if you wish on my GitHub.

    4. I’ll do as you say and wait for the code to soldify 🙂 the API will probably change a lot over the coming weeks/months.

    1. Nice, it is not as advanced but it is still pretty nice, thanks for the share 🙂 I might do an article on it when I get a chance

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s