SharePoint’s Founding Father, Jeff Teper, Talks About the SharePoint History, Vision, and Lessons Learned.

Hey everyone, I just wanted to share a
post from SharePoint's founding father and Information Week's 2009 Top 10 Innovator and
Influencer, Corporate
Vice-President Jeff Teper,
where he talks about the SharePoint History, Vision, and Lessons Learned.

SharePoint History

Hello!

As we are just two weeks away from
disclosing SharePoint 2010 at the SharePoint
Conference starting October 19^th,
I wanted to write three posts to provide context on the upcoming release. This
first post will cover the history of SharePoint. I hope it will provide some
useful perspective behind our vision and what we have learned as well as a few
fun anecdotes. The second post will cover the engineering process for
SharePoint 2010 - how we design and develop SharePoint in the Office team, what
new approaches we have taken during the 2010 development cycle and my take on a
few frequent questions I hear from customers and partners. The third post will
coincide with the opening of SharePoint Conference and cover the major feature
investments. After that, our team will start blogging in depth about the new
SharePoint capabilities. For folks who cannot wait, we have highlighted a few
of the new features on the SharePoint 2010 Preview Site as commented on several Office 2010 client
capabilities including a few points of SharePoint integration on the Office Engineering Blog.

Before we jump in to this post, I want to
thank everyone who has been with us on the journey so far and is supporting us
in the release of SharePoint 2010 and feedback for releases beyond. While I
will talk about the efforts that evolved into SharePoint, its origin dates to
around March 1998 when we first started planning the projects that led to
SharePoint. Since the first beta of the first release, we have been extremely
fortunate to have your support and hear your feedback. It has been extremely
rewarding to see all things you are doing with SharePoint. We have posters of
many of your sites on the walls of Building 16 on the Microsoft Redmond Campus
and these are a source of great motivation and pride for our team to take
things to the next level. We always appreciated your ideas and feedback
friendly or otherwise about where the product should go or how we could help
you better. We think SharePoint 2010 will be another big step forward but we
know there is more to be done and are thankful you care enough to keep pushing
us. Finally, we want to thank the Microsoft teams around the world who has
worked so hard to build and support the product with special thanks to a few
dozen people who were with us from the very beginning in 1998. We are very
fortunate to have a great team, customers and partners and that is what keeps
us fired up to come to work every day.

Ok, on to the post.

Pre-History

As long-time Microsoft followers know, in
the 1990s we had several efforts targeting information access and sharing. The
rise of web on the internet and intranet was a great catalyst to focus these
plans. I will highlight three threads that I consider part of the SharePoint
pre-history in the late 1990s. The first thread is focused on end users via the
FrontPage and then Office Server Extensions. These extensions installed on web
servers to let users create and edit web sites, post Office documents, participate
in discussions and get subscriptions. Many ISPs hosted the Server Extensions
helping millions of consumers and business users collaborate on the web. There
was strong integration with Office 97 and Office 2000 so you might say these
were the first cloud-based Office web collaboration releases! The second thread
focused on more sophisticated web site developers and was called Site Server
which was a grab bag of tools and services to help create and manage sites.
Finally, we had a set of internal incubations. One was called "TeamPages" which
was developed inside the Access team enabling users to create and edit simple
web-based lists. Under the covers, this was one of the first uses of XML and
was the genesis of the "CAML" markup inside SharePoint that pre-dated XSLT. The
other was called the "Digital Dashboard" which was initially an effort between
our marketing, Excel and SQL teams to make it easy to create dashboards in
Outlook but quickly moved to a web-centric design. We learned a lot about the
needs of end users, developers and IT from these efforts and the strengths and
weaknesses of projects that helped us design the first release of SharePoint
and make a strategic bet for Microsoft in the second release. This process
continues at Microsoft as we have both top-down planned efforts as well as
organic prototyping in our product and research teams. For example, our Office
Labs Group created a social networking project called "Townsquare" that we used
a bit inside Microsoft before refining the ideas in SharePoint 2010's new
social networking features.

Vision

As part of planning of what became
SharePoint(s) V1 and Office XP, we did lots of research around the world. We
visiting different size and type organizations and met with a wide range of
users, developers and administrators. We talked to lots of people and looked at
lots of intranets, document and knowledge management deployments. It was
immediately clear there was an opportunity to help both end users and IT be
more efficient and effective. Many customers told us about intranets or ECM
systems they had rolled out with lots of fanfare and then failed to get used.
The web site got stale. Users shared documents via e-mail vs. the official
repository. Many customers told us about their challenges with lots of
different silos that had grown up to meet the needs of a range of different
business units. One Fortune 50 company told us they had counted and had 600
different content management systems! Imagine the difficulty for users trying
to collaborate, developers trying to link business processes and administrators
trying to keep costs down and reliability up. Users told us they wanted to go
around IT. IT told us when they did this they create flakey custom web sites.
But as with many latent needs, there was no single definition of the ideal
solution in the industry. So we worked on a plan to address both usability and
manageability needs. As I was writing the blog, I went through some of our
early presentations and I was struck by a one slide PowerPoint deck we gave to
Bill Gates early in the planning. We knew reviews with Bill were always intense
so we decided to come in with a single slide explaining our vision with lot of
screenshots of what we thought the product might look like. We have lots of
fancy slides now to explain SharePoint but I thought it would be fun to show
what we shared with Bill exactly as is below. And yes it was an intense
meeting!

SharePoint V1 and Office XP

As some of you recall, there were two
SharePoints at the beginning. These evolved from projects codenamed "Office
Server" and "Tahoe" around the Office XP development cycle. "Office Server"
evolved out of the FrontPage and Office Server Extensions and "Team Pages" and
targeted simple, bottoms-up collaboration. "Tahoe" was built on shared
technology with Exchange and the "Digital Dashboard" and was targeted at
top-down portals, search and document management. They shared Office client
integration and "Tahoe" had web part and search links so top-down portals could
contain integrate with bottoms-up sites. Coming up with the name "SharePoint"
was a fun part of the process. We were in a giant conference room with dozens
of proposed names on the wall. Everything from meaningless words we would imbue
with meaning in the market to "Office Document Server" which was descriptive
but boring and limiting. We agreed on "SharePoint" in a single meeting but had
many discussions then and in the years about what were the right words around "SharePoint".
We eventually did elect to use the same name for both projects to convey the
integration and future strategy. We decided to ground "Tahoe" in the "Portal"
category because it was an area of a lot of buzz ("Portal Power" had was a big
cover story for InformationWeek, there were portal conferences, reviews,
budgets, etc.) and we had seen Lotus Notes and Exchange had struggled to define
a new groupware category. So "SharePoint Team Services" and "SharePoint Portal
Server" were the first names. The most memorable part of this release cycle was
the end game. I remember hours being in what we then called the "war room" (a
hyperbolic, if not insensitive engineering term that we generally avoid now).
Projected on the wall was an Excel spreadsheet with the status of 20 customers
we wanted to know were live with their deployment before we shipped the
software. Program managers would come into room with updates from daily calls
with these early adopters about issues, how much content was stored in the
system and number of active users. When everyone was green, we released and had
a fairly memorable ship party. As we still do today, we had screenshots of each
of these customers' portals papered to the walls so everyone in the team would
know we weren't writing code for ourselves but to help these customers solve
business problems. One of the most gratifying aspects of this first release was
the reviews. EWeek's headline was "SharePoint's a Hit" and Computer Reseller News
called SharePoint a "Whopper of a product". But we knew there was a lot more to
do to build a truly integrated product and experience between the two
SharePoint and we could do a better job of holistic planning across scenarios
and technologies. Since not many people see a server software box anymore vs.
just getting the CDs or downloads, I thought it would be fun to show the
SharePoint 2001 package:

SharePoint 2003 and Office 2003

While it was great to see the rapid
adoption of the first release of STS and SPS, it was clear customers wanted a
more integrated and comprehensive solution from us. As just one example, they
told us like they liked the WYSWIG HTML editing of SharePoint Team Services and
the Web Part declarative and reusable editing of SharePoint Portal but wanted
to use both models on the same site? Inside Microsoft, we were deciding how
much to bet on SharePoint. Would it be a good product in a broad family of
servers or an even more fundamental bet for the company? It was exciting period
because we were at an inflection point on both our platform and application
strategies. On the platform side, we were making a very fundamental bet on Web Services
including collaborating with vendors across the industry on the evolution of
XML, SOAP and much more. We saw this and the increased maturation and
scalability of SQL Server would give SharePoint a powerful foundation for
highly scalable web-based solutions that eased developer integration and
customization. With growing computing power, we could host even the largest
companies on a few sever farms (Microsoft has 3 for its worldwide SharePoint
deployment). On the application side, we were hearing customers wanted Office
to go beyond personal productivity to organizational productivity and we had to
decide whether Microsoft would invest in content management, portals, unified
communications, business intelligence and many other new scenarios. As you can
guess now, in classic Microsoft fashion we decided to bet big. However, we were
pretty quiet about until we were close to beta of the 2003 wave. The entire
Office team had SharePoint as a vision pillar and we designed the user
experience, architecture, extensibility and in a more holistic way than we had
done in the previous release. We evolved STS into a more scalable and flexible
platform and build SPS on top of it. One of the many heated discussions during
this time was between the WYSWIG and data-driven camps on web user interface
framework and we ultimately reconciled these in the SharePoint page model where
pages have zones where the web site owner can decide who can add web parts. At
this point, we also knew that customers, partners and Microsoft would be
hosting SharePoint on the internet and we made a lot of fundamental investments
in role-based delegation, partitioning, stateless front-ends, etc. to enable
this. These key architecture bets enabled us to ultimately offering SharePoint
Online to both dedicated and multi-tenant customers which we ultimately did in
the 2007 release. While this was a long release, we could not have gotten the
integrated experience and platform as lots of different skunk works efforts. It
really needed teams across the organization to work every day to research,
plan, develop and test together. At the same time, it was a very aggressive
project and we had to make a lot of tradeoffs about what to defer to the
following release. One of the most controversial was item level security. We
knew this was a very critical feature for some projects and a big checkbox even
for people who wouldn't use it. However, it was going to be huge investment for
us on top of everything else we were doing and was not realistically going to
make the 2003 schedule. So we decided to defer it until the next release when
we could do it right. We still use this as an example in the team of the tough
tradeoffs needed for software projects. One of the best bets we made in this
release was MySites. This was a classic bottoms-up feature from the development
team. In this era, most portal products allowed users to have personalized
pages ala MSN, Yahoo, etc. The team proposed we give every user a personalized
site and is still very proud they shipped it in 2003 before there was a MySpace
or Facebook. It is a great example we talk about in product development about
not always doing what the customer asks for explicitly but what we think they
might eventually find most useful. When we first released MySites in 2003, many
IT people first asked us how to turn it off. By 2007, they were asking us how
to turn it back on as their employees and even their CEOs were asking them for
enterprise social networking. Below is a picture we used many of times inside
and outside the team to focus the 2003 team on what I call "The Scalable
SharePoint Release". We launched it as SharePoint Portal Server 2003 built on
top of Windows SharePoint Services V2. There were many important lessons from
this project about the power of shared vision and a collaboration culture.
However, we learned while we provided IT a central infrastructure users where
users could go create their own sites, we had not yet provided all the
management tools and governance guidelines people needed. That and depth
category investments were the focus on SharePoint 2007.

SharePoint and Office 2007

With the integrated experience and
framework in place, we began to add depth in the various modules of SharePoint.
Hence I think of 2007 as "The Pie Release" because of the picture we used below
to describe it. Content Management was our biggest area of feedback and,
therefore, investment. As customers put more content in SharePoint and built
more sites with it, we got a lot of requests to take this to the next level -
more formal processes, more oversight and more sophisticated web publishing.
Building on the new Microsoft platform technologies such as Windows Workflow
and Windows Right Management, we invested document and records management
across the server and client. We were very aware customers had a wide variety
of ECM applications in place. So while we invested to be a leader, particularly
breaking down many of the user experience and programmability barriers, we made
sure SharePoint was an open platform and worked with vendors across the
industry on a variety of integration approaches including published APIs and
protocols. One of the biggest sources of feedback internally and externally in
2003 was the relationship between SPS and our Content Management Server (CMS)
2002 web publishing product which was used by a lot of internet sites. The
teams came together and proposed a shared experience and architecture that was
one of the highlights of the 2007 release. No longer do the pictures of the
SharePoint sites just sit on the walls of Building 16, you can see themselves
many of them live on the internet by navigating from a growing number of 3^rd-party
blogs and sites on the internet likes www.topsharepoint.com. This was another good example of the power of a
shared long-term vision. To be frank, we had a lot of pressure to do something
more incremental. Complementing content management, we increased our search
depth with a focus on new relevance algorithms and innovative people and
business data search. Finally, to enable customers to build business process
integration and business intelligence portals, we added Excel Services and
InfoPath Forms Services. Besides being exciting features, we gained invaluable
learning for the team how to have an architecture that worked in the rich Office
client and on the server with web access with high fidelity, round tripping,
etc. We have expanded on this and our experience with Outlook + Outlook Web
Access in the Office team for the new set of read-write Office Web Apps as part
of the 2010 wave. While customers appreciated the depth and management knobs,
from this release we learned we need to invest more in readiness (training our
employees, customers and partners) and frankly it took us a while to catch-up
with the demand of 2007. This blog was a big part of our investments that I
will discuss below. We are very focused on addressing technical readiness for
the 2010 release and the depth you will see around the SharePoint Conference
this month will be a big step towards this. However, there are many other
activities in progress to train 10s of thousands of people before we release.
We want to hear your feedback on what else you need so we can have the content,
training and industry ready by the release of SharePoint and Office in the
first half of next year.

Supporting SharePoint 2007

If you have been a frequent reader of this
blog, you know notice we have spent very few of the 100s of posts talking about
SharePoint 2010. This is for two reasons. First, we did not want to start
talking about the next release until it was solid and we were near something
actionable (e.g. getting educated on the product, try out the beta). Second, we
want to keep the focus of Microsoft and industry on great SharePoint 2007
deployments. Since the release we have made a number of investments our team
has talked about in this blog. I will highlight just a few things because many
have been posted about already:

- Enhancing Technical Content and
Guidance. Since the release of 2007, we doubled the size of the SharePoint
documentation team and increased the content on TechNet, MSDN and this blog. We
created a virtual team, called the SharePoint Content Partnership Council,
spanning R&D, documentation, consulting, support, marketing, etc. that
reviews your feedback and prioritizes the content to be written or improved. We
have focused on the key areas you have told us about such as interoperability
and governance. We also evangelized a lot of consultants and partners to post "real
world" experience to the blog since know that is very important to you.

- Increasing Microsoft and Partner
Staffing and Training. We were bullish on the adoption of SharePoint 2007 but
frankly we underestimated the demand and didn't have enough trained Microsoft
staff and partners at launch. Since then we expanded our internal support and
consultant staff significantly and introduced training programs reaching
thousands of people outside of Microsoft. We introduced a new multi-level
certification including a "Mmaster" designation and a packaged offering of 5,
10 and 15 planning for customers called SharePoint Deployment Services that
over a thousand organizations have used. This has been a great source of
feedback. We have already started the process of training Microsoft employees
on SharePoint 2010 and the SharePoint Conference starts the process for 2010
for the industry. Much more to come that you will hear about in this blog very
shortly.

- Creating a Customer Assistance Team
(CAT) - In the past, Program Managers in our development worked with our
consultants, customers and partners to get feedback and publish best practices.
In 2003, we created an elite team of experienced consultants called the
SharePoint Center of Excellence to drive this within our consulting
organization and partners. After the 2007 release, we went a step further
creating a CAT team within R&D that lived and breathed the most demanding
deployments to make sure our content on topics like capacity planning or
disaster recovery was deep and we had even richer feedback loop into the 2010
development team.

Enhancing SharePoint 2007

One topic some people ask us about is
responding to a wide range of requests for new features and solutions. I will
spare you the software laws of physics discussion about why we do not update 1%
of SharePoint every month. We spend a lot of time thinking about this and think
the model we have gets the great amount of capabilities out to customers in a
reliable and predictable way for a product of the scope and mission criticality
of SharePoint. However, we know there are places people need more solutions
with agility and I expect some people would be surprised by the investments we
have made so I thought I would walk through a few of them below.

- Cumulative
Updates - We heard your feedback
that you want something more convenient than hot fixes and more frequent than
Service Packs so we introduced Cumulative Updates for SharePoint and Office.
This was a big change for our development, testing and release process. It wasn't
all smooth at the beginning and we heard your feedback on this blog about
simplifying these and my sense is the feedback is now very positive.

- SharePoint Administration Toolkit - We created a team to address feedback on IT
operations requests and release supported tools that we have posted on this
blog. Unlike core features, these are more straightforward to do because they
don't invalidate the underlying code nor create a wide mix of configurations
for us to test.

- Toolkits and Codeplex
- As SharePoint is mission critical, you asked us to make sure the base product
is robust, performant, secure, localized, upgradable, documented, supported,
etc. Not surprisingly, this is expensive and causes us not to do as many
features as if we had a lower release bar. Some of you are fine with that.
Others have told us you would like some starter solutions to save you from
writing value-added code yourselves. Hence, we have invested in a broad range
of kits for SharePoint (Silverlight, Community, Site Templates, Patterns and
Practices, etc.) and fostered a vibrant open source community on Codeplex while
setting expectations these are not packaged products. I just checked and there
are now 876 projects for SharePoint. We know these are in various stages of
quality and adoption and unfortunately we can't bless them but do think it is
important to foster the community and listen to which ones you find valuable.

- SharePoint Partner Solutions and
Services - While we aspire to provide the most comprehensive solution in the
industry, customers have asked for a growing range of solutions and services
and we have worked to foster a growing community of thousands of business
around the world building on top of SharePoint. They have the expertise, focus
and reach to meet a large and growing range of needs. We also strive to make
SharePoint a great business opportunity to foster the investment and innovation
in our partners.

- Search Investments - We heard that great search is a critical part of
making a successful SharePoint deployment. We also see highly scalable and
flexible search technology as a key source of innovation across internet and
intranet sites, content management, social networking, business and
intelligence and more in the future. Therefore we did two things following a "good,
better, best" approach that people have liked with SharePoint. First, we
released a free version of SharePoint Search named Search Server Express.
Second, we acquired the leading enterprise search firm, FAST. FAST's ESP is the
most sophisticated and flexible technology serving the world's largest
publishing, media, commerce and telecommunications sites on the internet as
well as many of the most sophisticated internal search applications. The FAST
engineering team is outstanding balancing extreme technical depth with great
passion for the most sophisticated customer scenarios. We continue to invest in
both standalone products and products optimized for SharePoint (ESP for
SharePoint). Over the last year and a half the combined Microsoft Enterprise
Search Group has come together and we will share a lot of innovation in user
experience, relevance, connectivity and more as part of the 2010 disclosure. We
will continue to invest in standalone search technology and have some exciting
work coming in the next year there as well.

- SharePoint Online - Based on the SharePoint 2007 codebase, we released both a Dedicated
(single tenant) version of SharePoint Online as well as a Standard
(multi-tenant) version of SharePoint Online. This is all part of our Business
Productivity Online Suite including SharePoint Online, Exchange Online, Office
Communications Online and Live Meeting. The response to both the dedicated and
multi-tenant offering has been outstanding. Customers have told us they like
getting access to the most comprehensive and flexible set of collaboration and
communication tools with the reliability, security and manageability they need.
Large organizations such as Coca Cola Enterprise have adopted SharePoint Online
to give them more agility and free up their IT staff to partner with their business
users on more critical activities than configuring server farms. It also lets
us reach new business who previously were not able to deploy collaboration
technologies because of lack of internal IT expertise. SharePoint Online also
lets our partners work with us on a recurring revenue stream and focus their
business on higher value and profitable services. Finally, because Microsoft
does the operations and incurs the cost of running SharePoint Online it is an
incredible feedback loop at high scale that helps drive product and
documentation improvements that we will share with our customers running their
own servers. As I will share in the next blog post, the development of
SharePoint 2010 was very much informed by our experience during the last few years
on SharePoint Online.

So as you can see the team has been very
busy enhancing SharePoint 2007 and not just working on 2010.

Lessons Learned

To sum up our lessons learned from the
last decade on SharePoint that we'll apply to our second decade of investments:

- Customers come first. There is a lot of
exciting technology in SharePoint but the objective is not a building a
computer science project. What matters is solving business problems. We love to
see your sites so keep the case studies coming!

- Long-term vision and commitment.
Business is faster paced than ever and we and you need to continue to adapt. At
the same time, success usually only comes from a clear long-term vision,
commitment and feedback loop. I think that has been a key to your response to
SharePoint as we discussed above.

- Balance of both innovation and
execution. This is something we've prided ourselves on in Office. Sometimes it
means we incubate new technologies like we have done in the history of
SharePoint. Sometimes in means we take bets where we believe we can improve the
customer experience like MySites in SharePoint 2003 and the Ribbon in Office
2007. At the same time, shipping a product and service as comprehensive as
SharePoint Server and SharePoint Online is a critical engineering project and
you are depending on us to do it with rigor and continuous improvement.

- Viral and top down adoption. As
hopefully you have seen our approach for SharePoint dating back to the first
release is trying to strike the balance between empowerment and governance so
people can be more productive while the organization can manage its knowledge,
security and costs.

- Feedback loop. We need to continue to
listen to your feedback about what we've done well and where we can improve.
Even if we can't address the feedback immediately, we must always listen,
analyze and decide how we can best address it. We have a number of approaches
for this in Office and SharePoint that I will cover in the next post but we
always welcome your feedback in this blog and at the upcoming events.

Thanks for your support and for reading
this long. Hope it was interesting. I will post in a week about the development
process we used for SharePoint 2010.

Jeff

Jeff Teper - Corporate Vice President,
SharePoint Server, Microsoft