Publishing – Made Simple

“What happened to my publishing options?” is one of the frequently asked questions from sharp-eyed project managers who have just upgraded from Project Server 2003 to Project Server 2007. The answer is complex enough to deserve this blog entry.

My design needed to deliver in two areas:

- Simplicity, our research showed that very few customers understood the nuances of each publish option, especially when combined with the even more hidden Tools/Customize/Published Fields options

- Scalability, our larger customers were hitting bottlenecks because of the serialized nature of project publish.

Simplicity

To address the simplicity I decided to strip project publish to its bare bones, changing from the two dialogs below:

To the single option, placed next to the Save command:

It doesn’t get much simpler than that - this should make the publish option much less of a dice throw for most of our customers – however “power publishers” will be asking for more functionality, you should read on to the Power Publishing section for more information.

Scalability

History Lesson: Project Server 2003 used a background service called “Views Notification” that was responsible for moving data from the Project client sql tables to the Project Server and Views sql tables in the database. This process was serialized on a single project and the whole process was single threaded, using a single cpu (although later service packs allowed it to be run on multiple servers), as you might expect publishing rapidly became a bottleneck on larger systems. It also had other issues in terms of manageability, capacity planning and availability but more on that in other blog entries.

The new server architecture has radically changed how publishing works. Publish requests are placed on the Project Queue and are handled by a queue processor that:

• Is multithreaded and multi-server and serializes per project rather than across all projects
• Pulls data from the Draft database into the Published database through the MSP_*_WORKING_VIEW sql views
• Invokes the corresponding Report Publish that handles the transformation of the data into a report friendly format into the reporting database
• Optimizes (“folds”) multiple publish requests against the same project into a single request

Even with all the additional work done by publishing (such as moving all custom fields, and serializing data from our internal binary formats) internal tests have shown sustained publishing rates of around 1,400 projects per hour for non-trivial projects on a farm infrastructure.

And the Project Server 2007 queue infrastructure allows for much improved capacity planning and remote management as it exposes performance counters that the Windows System Monitor (fancy name for perfmon) and MOM can catch and track.

Power Publishing

The Project Server 2003 dialogs reflected the relative underlying complexity of the publishing process - much of that complexity was driven by the need to cope with the shared schema (now split across distinct databases) and the need to avoid the performance hit of a full publish. The server now publishes all changed information each time you request a publish operation.

Changes are tracked using revision counters on our primary entities and their children (for instance Projects own {tasks, assignments, specific custom field values, calendars and local resources} – these counters increment each time the project is saved, deleted rows are tracked in our _SHADOW tables.

The two main operations a power publisher wants to control are:

• When a team member sees a specific task assignment (aka “Phasing”)
• Who approves task progress (especially when the primary project manager is on vacation)

Both these actions are now controlled through the task sheet where the settings are now visible and editable (Yay!).

If you add the “Status Manager” and “Publish” fields to the sheet (as seen above) you can control the publishing process. Note that both these fields can be set as a group by filtering then using the mouse to drag the value down to more cells.

Firstly – use the Publish Yes/No flag to control whether a task assignment is placed in the Statusing (“My Work”) system – this flag can be toggled at any time & the project republished to make it active. This empowers you to publish a project a phase at a time, allowing you to avoid bombarding your team members with future assignments.

I made the flag at the task level because statusing is all about gauging progress against the whole task - assignment progressing is best managed in the Timesheet sub-system where work can be approved by resource managers rather than the project manager.

In the reporting database the flag is actually stored on the assignment:

 msp_epmassignments_userview.AssignmentIsPublished

There isn't a sinister reason for this, it just reflects the order in which we did the development work against a changing schema.

Note that if you toggle Yes-->No then the assignment disappears from the Team Members My Work (any approved work doesn't get lost though!) so use this power carefully! Also be aware that if the line is already in a timesheet it won't be pulled back.

Secondly - the Status Manager field has some strange rules (that echo those of project Server 2003) - it can be set to another pre-existing Status Manager on a task in the current project or the current user (ie the person with the project checked out and open) - this allows the value to be set even when the project is off line, and ensures that the Status Manager hhas (at least had) the ability to edit the project so that status data from team members can be applied once approved.

So if you are going on vacation you'd ask your deputy to open the project, filter on an appropriate time window and set themselves as the status manager for the tasks that will be active while you are out. On your return you can easily find those tasks and reset them back.

Phew! If you have any questions about other "mysteries of publishing", then please reply to this posting and I'll do a followup.

 

Technorati tags: Project Server 2007, Project Manager, Statusing, Publishing, Status Manager, Project Reporting

 

del.icio.us tags: Project Server 2007, Project Manager, Statusing, Publishing, Status Manager, Project Reporting

Comments

• Anonymous
  January 22, 2007
  Thanks for the much needed post. I have a question: In Project 2007 when you assign a resource to a task and  publish the project - by default the resources will get an e-mail alert. Is there a way to modify this behavior? Perhaps globally change the default on the server? We are trying to increase the use and acceptance of project server beyond the PM's and since every small modification to the schedule sends a whole new slew of e-mails - it borders SPAM. In Project 2003 there was an option of not e-mailing the members with task assignments and in our case the PM's started avoiding placing Resources in order not to bombard them with e-mail.

• Anonymous
  January 22, 2007
  Good follow up Uri, you can change the behavior using the task Publish flag as discussed in the article, until you set it to "yes" (the default) no notifications will be sent. Once you set it a notification will only go out when the task is changed.

• Anonymous
  February 27, 2007
  In 2003 you needed republishing when the owner of the project was changed and you needed to republish the assignment and check the option to become a manager. By what functionality is this replaced in 2007?

• Anonymous
  February 27, 2007
  In 2003 you needed republishing when the owner of the project was changed and you needed to republish the assignment and check the option to become a manager. By what functionality is this replaced in 2007?

• Anonymous
  March 14, 2007
  Our testing with the RTM version has shown that, even though setting Publish to "No" keeps the task of the resource's My Tasks list, the system still sends an e-mail notifying the user of a new task assignment.

• Anonymous
  March 29, 2007
  im getting error when publishing new project..failed to queue the message. any idea?

• Anonymous
  April 18, 2007
  Hello, We would like to know if the project manager can by any means set the "status manager" of a task of his project to someone else included in the project team. For the moment, only the creator of the project is available in the drop down list of the status field... Many thanks in advance for your answer!

• Anonymous
  November 26, 2007
  Hello, is there any chance to have task updates available to multiple "Status Managers" at the same time? By the way, if I change the Status Manager when there are some pending Task Updates, these are only available for approval by the new Status Manager, but the previous Status Manager sees their count on the PWA homepage... Thanks for all the information here.

• Anonymous
  December 27, 2007
  How do I insert queue time from projects?

• Anonymous
  January 02, 2008
  Erin - can you ellaborate on this?  In the project server world, items set in the queue as the server waits to process them - saves, publishes, task updates, etc.  Why would you want to increase this time? Zbynek - You can only have 1 status manager per task.  I'll look into the second issue you are seeing. Thanks, Heather

• Anonymous
  January 18, 2008
  Great posting, thanks for the info. I have a question though. When a Project Manager publishes its schedules very frequently, team members receive email updates continously. This not only confuses them, but also generates a lot of unnecessary traffic in the email servers. Another instance where I found this problem is if I need to delete a corrupt project and republish it, team members receive new notifications. This generates a lot of calls from confused team members, and is not desirable. I know change the settings for outgoing email prior to republishing a project that has been "fixed", but I know this is not appropriate. Is there any way I can disable email notification for a project temporarily? Thank you.

• Anonymous
  January 31, 2008
  When I try to publish a project it remains in the queue and does not get published. The message in the queue is waiting to publish. What can be the possible reasons of the project not getting published. I am login as administrator.

• Anonymous
  April 08, 2008
  The comment has been removed

• Anonymous
  June 18, 2008
  We really need the publishing functionality from 2003 back.  There are many times you want to publish for visibility on the server, but not send e-mail notifications to team members yet.  The spam is very bad and the "Publish" yes/no does not work (removes it from the team member's "My Task" list, but still sends them e-mail notifications which is even more confusing.  We may have to stop using the project server if we don't hear this is coming back in a future release.  Thanks.

• Anonymous
  December 01, 2008
  I second Virgilio Arciniegas' comments and request: When a Project Manager publishes its schedules very frequently, team members receive email updates continously. This not only confuses them, but also generates a lot of unnecessary traffic in the email servers. Is there any way I can disable email notification for a project temporarily? Thanks, CAP

• Anonymous
  December 16, 2008
  The comment has been removed

• Anonymous
  December 22, 2008
  I have many project plans each with many tasks, the person playing the Status Manager role has changes and I need to reflect this in the project plans. Is there a way to update all tasks with a new Status Manager instead of updating each task individually?

• Anonymous
  January 13, 2009
  Nival - you'll need to get the new Status Manager to open the project and set themselves as the status manager - once they have added themselves to at least one task you can then assign them as the manager of other tasks. Lots of reasons for this limitation (offline working, making sure the new task status manager has permission to open & save the project)

• Anonymous
  January 13, 2009
  Jon - you should be able to drag the task status manager value down using the mouse (SP1+) or if you are familiar with VBA then record a macro as you do it for one task and then add a loop for all tasks

• Anonymous
  January 13, 2009
  CAP & Mary - yes, we are working on improving this annoying feature of over-notifying the team member. Sorry it is causing you problems...

• Anonymous
  January 13, 2009
  The comment has been removed

• Anonymous
  January 13, 2009
  Sudeep - the Windows application Event Log on the Project Server is the best place to start diagnosing this. Make sure you have the latest updates installed. You can try using Server Settings/Deelete Enterprise Objects to delete the project from the Published (and Reporting) database and then republish the project. Be aware that doing this deletes you transaction history in My Tasks/Task Status Approvals...

• Anonymous
  February 05, 2009
  Hello, Publish is grayed out in our copy of Project 2007, what do we need to do?

• Anonymous
  February 06, 2009
  For publishing to be enabled, you need to ensure a few things:

• You must be connected to the server. Check the status bar to see if you are disconnected.
• You must have initially saved the project.
• You must have permission to publish the project. Check with the server administrator if you are unsure about this. Thanks.

• Anonymous
  February 25, 2009
  Using Project Server 2007 with the Infrastructure Update, we set up an automatic approval of task updates. Is there any way to set up an automatic publish after a task update is approved?

• Anonymous
  March 18, 2009
  I am currently accepting and approving weekly updates from over 130 projects and then publishing them on Project Server 2007.  The save, publish, and checkin sequence for each project consumes 5 minutes of my time.  Thus 25% of my week is being used just to publish projects.  Is there a facility to dispatch the publish sequence to a background process and free me up to deal with other work?

• Anonymous
  March 19, 2009
  I agree with Uri's original post, that the email sent due to the publish function is spam. I disagree with Patrick's responce that notification will only go out when the task is changed. The tool has a bug where all resources for all published task assginments get an email. This includes email to resources on completed tasks that have not changed. For users that run data analysis reports, you must now publish the file after every PWA task update to get the actual hours into the published & reporting databases so it is included in the building of the cube. I would like to see a fix for email generated by publishing to be limited to change only. Additionally I would like an option to automaticall publish task updates or have a way of publishing all files on the server so data analysis reports at the portfolio manager level have accurate data that matches all task updates performed by status managers. It would also be nice to disable email when performing a publish all files.

• Anonymous
  May 18, 2009
  Patrick - Is there a way to automate the publishing of updates received from team members? I have used rules to automate the approval process, and wish to have these updates published instantaneously, without having to publish every project manually. I should be most grateful for your assistance.

• Anonymous
  May 18, 2009
  Michael - great question, we don't do autopublish for you, however you could do what we do on our internal system and run a utility to publish projects every night (ping me at pconlan (at) microsoft (dot) com if you want some code to do this. If you want it more real time then you can write an event handler to consume the Statusing.ApprovalsUpdated event to publish after approvals...

• Anonymous
  May 18, 2009
  Hugh - se my response to Michael, lmk if you want a copy.

• Anonymous
  May 18, 2009
  Bill - the recent SP2 release should have improved things (sorry if you felt that we weren't aware/taking this seriously) - LMK if you don't see improvement by posting here (as others will get the bad news as well) cheers....pc

• Anonymous
  May 18, 2009
  Phaedrus - see my respoonse to Michael, to do this more real time you'll need to hook up event code. Cheers...pc

• Anonymous
  June 02, 2009
  Patrick, we've written some code to publish projects and we got some problems with a cube. The work done by resources are not updated in the cube until project is published by Project Professional. What's the differece between publishing in Project Pro and in the code? If you prefer, we can e-mail you with mode details about the error. Thanks in advance. Leandro.

• Anonymous
  June 03, 2009
  I have a draft/working copy of a schedule has become corrupted. Is it possible to replace the draft copy with the copy of the schedule in the published database?

• Anonymous
  June 03, 2009
  Leandro - Pro publishes using the same api that the server users (Project.ProjectPublish), after: 1/ the team members have submitted their status, 2/ and the task status manager (usually the project manager) has approved the work, 3/ and the project is checked in (so we can put the approved work onto the project) ==> then the data should just move. You can use the approvals history to check that the work made it onto the project ok... DFK - yes (and no) - you can open the Project from the Published database in Pro and then do "Save/As" to save it as a new project - once you've checked that this is ok you can then use Server Settings/Delete Enterprise Objects to get rid of the old project from the system (pro tip: make sure you have a good SQL backup before doing this, just in case you have finger trouble) The issue is that the new version will have different UID for all of its objects - you may not care. What is more painful is that any team member task status not already approved, applied & published will be lost :-( One option to add an extra insurance policy is to use the automatic archive functionality in the server that copies changed projects from the Draft DB into the Archive DB - in  this case you could restore an older copy and retain all the old UID and not lose unapplied task status data. Do you know why the project went bad? I'd recommend opening a case with CSS if you can. Thanks again for the comments...pc

• Anonymous
  June 04, 2009
  Hi Patrick. Thanks for your answer. I'll try to explain a little bit better: we've developed a new form to users fill the actual work. Then we configured the auto-approve for the project manager. In the code, we submit the actual work to the project server, publish the project and then check the project in. To publish we are using the QueueSystem.QueuePublish method. The project is published, but we get the problem that the data is not updated at the cube. Do you have any idea about this issue? Thanks again. Leandro.

• Anonymous
  June 05, 2009
  Leandro - troubleshooting: 1/ Log on as the Task Status Manager and check the update history or drill into the task history from the task detail view on the server - did the work apply correctly? 2/ After autoapplying the work and publishing on the server, open the project from the published db before opening from the draft db - does the work show? ...pc

• Anonymous
  June 11, 2009
  I am still having issues with the email alerts sent related to the publishing function. Notifications are sent to the assigned Resources on all tasks with the publish toggle set to 'Yes' regardless of whether any changes have been made to the task. If the project is published again and no changes have been made, all resources with published tasks will get an email alert. I have applied SP1, Infrastructure Update and SP2 to both Project Server and Project Professional on our Project Manager's workstation. The user alerts are set to only receive alerts when they have been assigned a new task or their existing tasks have changed. Is there a work-around or hotfix for this issue?

• Anonymous
  June 24, 2009
  The comment has been removed

• Anonymous
  March 04, 2010
  Can we setup automatic re-publish (Customize it to every hour or day,etc) through PWA in MS project server 2007?

• Anonymous
  April 07, 2010
  A Status Manager is set for tasks on an Activity Plan. Apparently changing the Owner of the Activity Plan does not change the Status Manager. Nor does there appear to be a way to display or change the Status Manager on an Activity Plan task. Nor does there appear to be any way to report (in the Reporting Database) the Status Manager for an Activity Plan task. Is there a way to report the Status Manager for a Project or Activity Plan task? Is there a way to change the Status Manager for a Project or Activity Plan task?