OneNote 12 - Working as a team with shared notebooks

First, let me point out that Owen Braun is blogging now. He's the lead program manager on my team responsible for OneNote. He's got a great post up about how we now support multiple notebooks. And he's promising a whole feature on hyperlinks...

Recently I've had some mail from various OneNote users telling me that they are using OneNote for team work - shared sections and folders of notes which every member of the team can read and contribute to, and of course search. I've also been reading blogs like this one. I like this post because sbreck describes exactly the type of scenario we are targeting OneNote 12 at. Of course, he is trying to do it with OneNote 2003, so he has a few issues. It works, but it runs into limits. OneNote 12 has had significant work done to make it a breeze to work in small to medium groups effectively and quickly. Allow me to drone on….

How did we get here?
A lot of people view OneNote as a single-user application designed for personal notes. This is a pretty narrow view of an application we have much grander ambitions for. Of course it is not as narrow as the "it only works on TabletPC" myth (grr), but it's still pretty narrow.

As we started planning OneNote 12 in the summer of 2004, for kicks we dusted off our original vision for it. Right there in black and white in the original vision doc from early 2001 there is a sentence I wrote at the urging of my dev manager to specify what we were NOT going to do in the first release as a guide to the team. It states, "Scribbler [OneNote] will be a single-user application for personal use. Although we intend to make it a multi-user application in the future, our first release will focus on the single user scenario to make sure we get it right".

Our dev team was actually very excited to design into OneNote from the very beginning the ability to treat every page and every little bit of content on a page as a node on a graph that can be modified by any number of user agents. They did not want to make any assumptions in the architecture or the code that required a single user to be doing the modifications to this graph in a particular sequence. So we had to put in a little explicit "Down Boy!" to make sure we stayed focused for the first release.

When we had a chance to do SP1 with a load of features, we enabled real-time multi-user editing of pages. We had already found OneNote a compelling way to enhance meetings - it would be even better if we could all use it together - including people attending by phone. We hooked ourselves up to a convenient peer-to-peer sharing technology that was in every version of Windows we supported: DirectPlay. This is used for multi-player games on LANs, but it also works for note taking apps that work using the same principle of passing changes to the other members in the "game'. I wrote more about this here.

The upshot was that we had a non-trivial multi-user editing surface which could be used in real-time. Multi-user editing of a document is one of those hard problems in computer science. It's been done for simple documents such as text files or pixel-drawing whiteboards, but to do it for something tricky and complex like a OneNote page was novel. I am not aware of any non-research application out there that does this to the degree we do (where any user can edit at once in an unstructured way). There were a few limitations to this approach - namely it was really focused on synchronous multi-user editing (i.e. you are all working together in real-time), so it was not a good candidate for collaboration that lasted more than a few hours.

Another thing we added in SP1 was the ability to open a folder on a file share. We had some customers early on (even before we shipped the final code for the original release) who immediately saw that OneNote was a great place to keep the notes for a team (of, say, lawyers and paralegals) for one or more projects. They had a problem today where only one person could read and access notes and documents at one time with their existing tools, and they needed to move faster. They were quite motivated to make it so each user could see and add to or annotate a shared set of information, such as the case file for a client. It's sort of interesting to have a PDF file in a shared location such as a file share or document management system. More interesting is if you can see the other lawyers' opinions of the various clauses in that contract or brief - written in ink or typed beside the pages, or highlighted passages.

OK, still with me? Here's where OneNote 12 comes in.
OneNote 12 allows something we are tentatively calling "shared notebooks". Other working titles include "Nirvana and you", or "Happiness on a stick". [:)]

With a shared notebook:

• You can work by yourself or with others - it is asynchronous
• You can come and go as you please - there is no "session" to reconnect to, and no duplicated pages when you reconnect
• Everyone can edit at once
• Everyone or anyone can be offline (no net connection) and still edit
• Offline edits are synced and auto-merged with the shared notebook when the user gets back on-line (even several weeks' worth of changes)
• Changes are automatically propagated to a server and thence to any of the clients involved in the shared notebook
• The contents can be searched instantly since they are cached locally and indexed.
• Team members can be added at any time and their machines will sync up to the latest bits.
• Everything written is tagged with who wrote/modified it and when it was created/modified
• You can query the notebook to show you what was written since you last looked at it - the actual text gets highlighted.
• Creating a shared notebook is as simple as creating a new notebook and picking a file share or SharePoint (or any WebDAV-capable server) for it - we'll pick a decent default as well if you don't care. If you make a notebook shared, we'll offer to send an email invitation to the team members you specify and that's all there is to it.

Shared notebooks may sound fancy but really they are just the most obvious thing. I can work on page as I would with my private notes. So can you. You can see what I wrote, and vice versa. So can Hans, Reza, Mary and Yuriko. We can continue this way for months if necessary. It's the most natural way to work together. It's like sitting around a big desk with your team and having a space in the middle where there are post-its, documents, notes etc. and you can see what others are doing or have done while you stepped out. There's no barrier to grabbing a doc and commenting on it while some one else writes the conclusion. Or add comments to a post-it note already on the desk while someone else is reading it.

Best of all you can go on a trip and take the whole desk with you for reference, make changes, then when you come back through the magic of software your modifications are just merged in with the work that has been going on. (Ok, you can see that the physical metaphor sort of runs out of gas because this is something you really can’t do without software.) In software there's no bumping into each other and holding a paper at an awkward angle while someone else tries to see it. Its also trivial to search and locate anything in the massive archive you can build up - very unlike paper.

Today when a team works together they have a file share or a SharePoint site where they put files. You can browse to that place and see what files are there. You can't see what's in the files without opening them. You can't see what others think about each file. You can’t access these files offline unless you take them offline manually and lock others from editing them to avoid nasty conflicts when you go to put your changes back. All those barriers go away with OneNote 12.

When I try to describe this in words I usually get a few questions:

1. Won’t this be chaos? No, it won't, because we've done it and it isn't. Just because you don’t have barriers doesn't mean that people go insane and start modifying everything everyone else is working on. What happens is that everyone is able to READ what other people are contributing on different pages, and maybe also comment on it.
2. Isn't this just a wiki? Yes and No. A wiki is great because of the same sort of thing - everyone can read what others have added and modify it if it is wrong or incomplete. But a wiki still forces you to get a write lock on an article or page unless you want to deal with the risk of it getting modified while you are making your changes. And then merging the changes is a manual process if that happens. And wikis do not support offline - that's a killer benefit and in fact the really hard part. Not to mention that the OneNote writing surface is much richer than a wiki: pictures, annotations, drawings, ink, embedded documents, etc. Wikis are also hard to get a handle on since they can't be "organized". It turns out that having the notebook structure available is very important to show people what is available and how to use the space.
3. What if I don’t have a server? Any file share including one you make on your own PC will work. Unlike a wiki, the other members do not have to be able to reach your PC in order to work on the shared notes. As long as you are both on the network at the same time periodically the changes will be synced and merged.
4. Does this mean I can edit a Word document or PowerPoint with other people at the same time? No, we didn't enable this capability in other applications - they weren't designed for it from the ground up like OneNote was. You can of course put a Word doc or PPT file in the shared notebook and it will be replicated to your teammate's machines like any other content. They can edit the file and the changed document will be synced back to all the machines. There's no protection against conflicts though, so be sure no one else is going to modify that document.
5. Is this based on Groove/LiveMeeting/Office Communicator/SharePoint/Windows offline files? No. All of those technologies more or less work at the file level. From the end-user perspective they move a file around (not SharePoint, which doesn't move files except when you edit/read), just as Shared Notebooks move Word docs (see previous question). Groove does some bandwidth optimization to move only diffs of docs, but this is below the user level. If you have modified a doc in two places you are on your own. None of these technologies will solve a conflict for you that is inside a document. These technologies are not focused on letting several people edit the same content whenever they want to without regard to what others are doing. (note: A partial exception is Groove, which has a few tools that let you make lists with each other, edit discusisons, scrawl on a simple whiteboard, etc and changes to these are merged automatically. But there is no Groove tool yet for collaborating this way on a document surface - basically because it is really hard to do even for one app, let alone generically for arbitraily complex document types. Maybe in the future now that Groove is a part of Microsoft we'll have some cool synergies...) BTW, of course you can host a Shared notebook on a file server or a SharePoint site - OneNote handles the offline caching, syncing, and merging.
6. Isn't this just going to cause conflicts all over the place? Um, no. The reason is that we have some clever merge smarts and we also get to take advantage of the scenarios OneNote is used for (vs. say Word or PPT). Usually when you sync you have only modified an area no one else has modified, so the merge is straightforward. If you add a page to a section in a shared notebook, it can be easily merged into that section even if others have added or modified other pages. Likewise if you add a picture to a page that others have edited, that can be slipped onto the page without bothering anyone. If you added item 3 to a list of two items and someone else added item 3 as well before they saw your change, we just make one of the items #4 and continue on. If you delete a word and someone else modifies it, we just say that modifying wins over deleting (partly because deleting is easy to do again, but recalling the modification is harder). The only times you actually see a conflict is when there are two content modifications to the same paragraph - maybe I changed "great" to "swell" and you changed it to "awesome". In that case we will pick one winner (the first one to update the master) and then show a ghosted page that indicates where the conflict edit is. In practice this basically doesn’t happen if you are not offline, and even then it is very rare to work on the exact same text in the same paragraph as someone else - you tend to self-police this. And that is more of a Word scenario where you are "polishing" content and wordsmithing it. It's more likely with OneNote that you are just pasting in content from the web or other documents, adding a picture, building you own list, etc.
7. Do I have to invite others to the shared notebook - or can I just "collaborate with myself"? Bingo! In fact this is the answer to working on two or more machines with OneNote. You can just set your own personal notebook up to be shared and replicated across as many machines as you own. And that's all there is to it.
8. Ok, how does this really work? Every copy of OneNote12 keeps a local cache of all the notebooks it is aware of. Periodically OneNote looks to see if the server share where the shared notebook is located is available, and if so, is the notebook there (the master) updated since it last checked? If there is an update, OneNote downloads the part that has changed and applies it to its local copy. Likewise if you make a change, we now autosave to the cache (way faster than autosaving directly to the server!). Periodically we try to push up the changes to the server, again just moving the part that has changed if possible. Until we successfully update the master, we keep around a "baseline" which is a version of the master as it appeared when we last synced to it. That way we know what changes to push up, and more importantly we know what to do when the server copy has changed as well. Once a sync completes successfully we update our "baseline" to reflect the new "truth". Obviously if you are offline we can't sync, but you can keep working merrily along and we'll sync everything later.

Ok, that's fine, but what use is this?
There are lots of uses as I've mentioned above. An example for us was when we were trying to get our plans firmed up for the last milestone of development. Usually that process entails putting specification documents on a server and having devs read those docs and make private comments in a non-shared place. We usually try to have at least two separate devs estimate the work to develop the proposed capability to cross check and make sure nothing is missed. The downsides of this process are that no one can see if the devs have started reading the specs or making comments. No one can see the comments. Devs have to ask questions via email or physically locating the PM, and no one else can see those questions.

For our last milestone we used OneNote 12 and created a shared notebook, using several sections but one especially held a page for every proposed feature. We put a table at the top of each page, and embedded the spec document in question. The table also had a place to list the devs, testers, comments, unanswered questions (marked with note flags), etc. Everyone on the team could see the notebook at any time, even on the bus ride home. You could see which devs had started making comments on which spec. Two devs could comment on the same spec at the same time. We could query the whole notebook to see how many unanswered questions there were and what they were.

There were some neat side effects too. For example, previously we used to put these specs out and dev would say they would estimate them. Now we could actually see that they didn’t start estimating them until the following week (with an integrated development team it is gold to know what is really happening, not just what is supposed to be happening). Some PMs took advantage of that knowledge to put up an updated spec that had more detail - something we had been asked not to do in the past as devs had often started estimating the spec unbeknownst to us.

The most amazing thing was that we were done with the whole exercise and had higher faith in its being well executed in a matter of a week - usually it takes a month. This underlines one of the selling points of this approach - it makes your organization more agile. I was talking to a law firm the other day which is interested in OneNote. I asked them why a firm that charges by the hour is interested in time-saving productivity tools? The answer: law firms that engage in litigation are more interested in winning the case than in hourly fees. Anything that allows them to put a case together and move faster than the competition increases their chance of winning. The CIO told me that he learned something from a General when he served with the military: winning is all about not waiting your turn.

Comments

• Anonymous
  September 26, 2005
  
  Soons so useful... when can I get it? When is the beta... or how about a CTP?

• Anonymous
  September 27, 2005
  ummm, WOW. Nice. Again, I look forward to see this in action.

• Anonymous
  September 27, 2005
  Chris:
  
  As always, I'm most impressed. Keep up the great work!

• Anonymous
  September 27, 2005
  The concept here is amazing. I have a tablet and feel so much more organized with the current version of OneNote. I understand that you don't need a tablet to use OneNote, so I really hope this catches on in our work place.
  I think that without a tablet users often don't bring their laptops into meetings (they instead use paper). I know that those folks will not want to scan their notes into the computer and OneNote after every meeting.
  If everyone in our company carried a tablet pc to meetings and used the next version of OneNote we would see significant improvements in so many areas where we rely on other slower methods of communication.

• Anonymous
  September 27, 2005
  OneNote 12 sounds amazing! I desperately need the background OCR and the shared workbook (to share with myself, thank you very much). Please release it soon!

• Anonymous
  September 27, 2005
  The comment has been removed

• Anonymous
  September 27, 2005
  So, is anyone blogging on the changes in 2003 SP2 or is it just bugfixes.
  
  \uslacker

• Anonymous
  September 27, 2005
  The comment has been removed

• Anonymous
  September 27, 2005
  Chris, thanks for the link. Looks like I will be upgrading myself and my team to OneNote 12 as soon as it is available. I take it the "shared session" concept is now integrated into the full-time operations of the OneNote file when it recognizes it to be in a shared location?
  
  uslacker, I blogged on SP2... it's two bugfixes. Nothing exciting.

• Anonymous
  September 27, 2005
  Chris -
  
  Excellent! This pleases me greatly. (I also like your approach of spreading the word of features ahead of time, to build up community interest and excitement.)
  
  Is there anything that I should be doing now to enable my organization to make use of this? What kinds of WebDAV servers will work with OneNote (other than SharePoint)? Will there be any new API's available, i.e. what kinds of data sources/sinks can we plan/speculate on integrating with OneNote?
  
  Keep up the good work! (Include me in betas!) Good luck as you get closer to release!
  
  
  

• Anonymous
  September 28, 2005
  The comment has been removed

• Anonymous
  September 30, 2005
  I have OneNote installed, but to be honest with you, I honestly don't use it a lot. Its a nice application but I truly believe I would be more productive with it if I owned a Tablet PC which I won't be investing until Windows Vista hits RTM. What I will do in the mean time is put a little more effort into using it to copy little bits of info that interest me on the web and past them into OneNote for future reference. I still believe though that OneNote and Tablet PC are bread and butter.

• Anonymous
  October 08, 2005
  The comment has been removed

• Anonymous
  October 31, 2005
  If you want shared editing right now, get SubEthaEdit. This is a Macintosh app which won the Apple Design Award in the student category about two years ago.
  
  Also, if you'd like to see what's coming up in One Note, check out the Mac apps it's copied from: NoteBook by Circus Ponies Software, and NoteTaker from AquaMinds. They're both derived from an earlier NeXTSTEP code base.
  
  And, if you'd like to get software features now which won't make it to Windows for at least three years, the solution is to get a Mac.
  
  MS: reinventing the wheel (poorly) since 1975. ("Roundness" feature deferred until post-longhorn.)
  

• Anonymous
  December 06, 2005
  The difference between MS and Apple... MS is actually able to deliver products to the market. Oh ya, and Apple apologists whine a lot more.

• Anonymous
  December 06, 2005
  J. Random Poster: In case you're interested, I wrote long ago (my second post ever) about how OneNote got started:
  http://blogs.msdn.com/chris_pratley/archive/2004/01/30/64898.aspx.
  
  BTW, current OneNote 2003 already has real-time shared editing (in a full 2-D text/ink/graphics space - not just in a 1-d text doc). See this post: http://blogs.msdn.com/chris_pratley/archive/2005/02/22/378667.aspx

• Anonymous
  January 18, 2006
  FYI, a prototype for an AJAX-based online/offline wiki: http://blog.monstuff.com/archives/000272.html
  
  This approach still takes a lot of work and needs to be improved, but it shows some promise.

• Anonymous
  January 19, 2006
  The comment has been removed

• Anonymous
  January 28, 2006
  Chris,
  
  I'm loving OneNote 12 more and more each day. I experimented with "Sharing" today and just loved it. I'm trying to find a "step by step" way I can share my notebooks with myself. I have it hooked up on my home network, but I want to be able to get my office computer to sync up with my home desktop over the internet. Where can I learn how to do that?

• Anonymous
  January 29, 2006
  Let me clarify a bit (when I read my post later I couldn't make sense of it-I hate it when that happens!).
  
  I am able to keep notebooks "in sync" on my home network via the shared notebook option. Yesterday, I experimented with live sharing sessions--I installed ON12 on my desktop at work and was able to use it everywhere I had internet access.
  
  What I'd like is to be able to keep all three (desktop office, tablet, desktop home) in sync. I think that is one of the original visions of ON (and it is a perfect vision, IMO). What I need is a step-by-step way to do it.
  
  If sharing to WebDAV is the way to go, I'd like a little bit more info on how to do it and what the possible downside is.
  
  For what it's worth, I deeply thank you and all the OneNote12 team--this is an incredible resource that I am trumpeting to all my students, faculty and friends. They are usually all amazed at what it does.

• Anonymous
  January 31, 2006
  Philip, thanks for the very kind words. You can sync your work machine with your home machines in one of two ways.
  1. choose an internet server to host your notes. This can be a web site or "hard drive in the sky" site that supports WebDav (i.e. directly saving to that site). Then choose open notebook" from OneNote 12 and open a folder on that web drive as a notebook. Put everything there (even more notebooks if you want beside the one you made). Note that this does NOT work in OneNote 12 Beta 1. It will be working in beta 2.
  2. pick a machine which all your machines can reach via network or VPN. In most cases that will be a machine at work (either your own machine or a server) since I suspect you can't VPN into your home network (if you can, great). Make the location of that note book or notebooks a folder share. Then use "open notebook" on that folder from all your other machines.
  Either way, you can work on your notes as much as you like even without a connection. Syncing will happen when each machine can see the shared location (might be only when you VPN into work). Check out my earlier post on using OneNote 2003 on two or more machines - some of the concepts might be clearer even though this does not discuss Onenote 12: http://blogs.msdn.com/chris_pratley/archive/2005/06/25/432556.aspx
  

• Anonymous
  January 31, 2006
  Thanks for the answer, Chris. For some reason, I thought the WebDAV option was available now; so I've got a little more time to figure out the best WebDav product for me. Feel free to make a recommendation! And again, great work on ON12.
  
  plc

• Anonymous
  February 15, 2006
  Naming baby!Today our branding folks finally unveiled the naming for Office 12 (which includes OneNote...

• Anonymous
  February 18, 2006
  Beta 1 of "OneNote 12" (and "Office12") is now available for download for those people who are signed...

• Anonymous
  March 18, 2006
  The idea of collaborating with myself is great. I've been using a file-share and putting the notebooks on my portable. Works, but isn't that great.
  http://www.thenotebookcomputer

• Anonymous
  March 18, 2006
  Luke, are you using 2007? If so, when you say "isn't that great" can you elaborate?

• Anonymous
  March 29, 2006
  Does this feature include version management, or is this best done by the server, i.e. Sharepoint?

• Anonymous
  March 29, 2006
  Naadir, if the place you store the notebook has its own versioning (e.g. sharepoint) then you can use that. If not (e.g. a file share) then OneNote allows you to control how many versions back you would like to keep locally on your PC. The default is 2 but you can adjust that to be any number depending on how much disk space you want to use.

• Anonymous
  May 26, 2006
  How do I turn this offline caching thing off!!!! My One Note crashes since the offline cache file grows over 5 GB!!!
  
  I am using Beta 2 on a tablet PC 2005, and have a lot of Audio and print files.  This offline cache file is killing me! in disk usage and crashing.  I went to Options and do not see how I can disable this "feature"

• Anonymous
  May 26, 2006
  Ed, send me a mail at chrispr(at)microsoft.com and we'll take a look. The cache can't be shut off since it is integral to how OneNote 2007 works.

• Anonymous
  June 07, 2006
  Its an obvious thing. You have two or more PCs that you work with. You want to access all your stuff...

• Anonymous
  June 07, 2006
  Its an obvious thing. You have two or more PCs that you work with. You want to access all your stuff...

• Anonymous
  July 27, 2006
  Chris, if we could talk, re vision for notes across many pc's
  949-305-0707 home
  949-412-2708 cell
  

• Anonymous
  October 05, 2006
  If you are not familiar with creating Outlook tasks flags from OneNote notes and synchronizing them,

• Anonymous
  December 03, 2006
  Its an obvious thing. You have two or more PCs that you work with. You want to access all your stuff

• Anonymous
  December 03, 2006
  Naming baby! Today our branding folks finally unveiled the naming for Office 12 (which includes OneNote

• Anonymous
  December 03, 2006
  Beta 1 of "OneNote 12" (and "Office12") is now available for download for those people who are signed

• Anonymous
  December 03, 2006
  Last week we released Service Pack 2 (SP2) for Office System 2003. This includes OneNote 2003. Although