Tuesday, November 10, 2009

Visio Services

Visio Services is a new set of services added to SharePoint 2010 that let you:

1) View diagrams in the browser without needing Visio on your machine
2) Refresh data-driven diagrams in the browser
3) Integrate diagrams into SharePoint applications

What makes it so compelling is that the diagrams can represent data, but not simply static data, rather data that is constantly being updated. That means that a diagram can be displayed in the browser while representing a real entity such as a network or process and see changes updated in real time. Thus we have much richer way to display data than spreadsheets and charts without the long development times of a Silverlight or other rich visualization tool. The diagram can be created once in Visio using a data source such as a SQL Server table, get published to SharePoint, and begin immediately reflecting changes as the underlying data source changes.

Some of the key features are:

1) Browser-agnostic (yes, it even supports IE)
2) Full-fidelity Visio diagrams if your system supports Silverlight
3) Without Silverlight support, you get a PNG image of the diagram, but it still has the same data update capability
4) Supported data sources include: SQL Server, Excel, SharePoint lists, OLEDB, ODBC, and Custom sources such as flat files and XML
5) Can embed Visio Services web parts directly into web part pages
6) Visio services can be tuned to your specific hardware characteristics
7) Requires Visio 2010 to create the drawings

One good place to learn more about Visio Services is the Microsoft Visio Team Blog: Introducing Visio Services

Saturday, October 31, 2009

ECM Sessions for SharePoint 2010

Rather than spend time re-summarizing information that is available elsewhere, I'm going to point those interested to some of the best session summaries. In addition, I'm going to continue summarizing the SharePoint conference summaries that I can't find elsewhere and that I think contain significant material.

1) To understand the Microsoft KM approach with SharePoint 2010, the following articles summarizes their experience with implementing KM on SPS 2010: Building an Enterprise Knowledge Management Solution in SharePoint 2010

2) I've already blogged this session (ECM for the Masses) which was an excellent introduction to SP 2010 ECM features, but this article on CMS offers another perspective that you may find valuable: ECM for the Masses: How SharePoint 2010 Delivers On The Promise

3) Here is another summary of the new Document Management features in SharePoint 2010. It's brief and to the point: Document management Sharepoint 2010

4) To get a balanced perspective on the new Records Management capabilities of SharePoint 2010 check out The Scoop: SharePoint 2010 Records Management

5) If you want to understand Sandboxed Solutions, the preferred deployment method for solution packs in SP2010, read this: SharePoint Sandboxed Solutions

Here's something new about content types. In SharePoint 2010, content types are now service-based rather than being bound to a site or site collection. Content types will automatically be available in all sites throughout the farm, so there is no need to deploy content types to separate site collections.

Monday, October 19, 2009

SharePoint 2010 ECM for the Masses

The presenter set the stage by declaring that 50-60% of ECM implementations fail, primarily because of lack of user buy-in. SharePoint 2010 addresses many of the key areas where this buy-in has failed in the past.

1) The new Document Center, which is intended to be a knowledge repository has views for:
a. Newest documents
b. Highest Rated Documents

Document ratings can now be turned on for any library through a built-in feature. The document rating system allows users to rate documents by clicking stars. You can also find out who gave the rating so that you can determine how much weight to give it. The rankings are displayed in the search results and rankings feed directly into search relevance. Items that are ranked higher appear higher in the result list.

One innovation that will make integration with other LOB systems and ECM systems easier is the unique Document ID that is now assigned to each item and remains with the item no matter where it is located in the site hierarchy. The document ID can be easily configured to use specific prefixes. You can also plug in your own number generator.

Managed Metadata

The key new feature for ECM deployments is the ability to centrally define taxonomies that can be leveraged within and across farms to classify and find content quickly. Managed metadata is now a centralized service that provides the following features:

1) A metadata tree derived from approved taxonomies which allows users to zero in on the document they are searching for. The metadata treeview can be filtered by built-in filters (and custom filters can easily be configured). All tags and modified by date are included OOB.
2) Automatic extraction and promotion of metadata from images uploaded to the central Asset Library.
3) Metadata can be used both in searches and in refinements, which is a set of facets that allows users to drill down on categories.
4) Folders can be configured to automatically tag content with specific metadata, i.e. you can have the metadata Month="December" and Year="2009" to everything in the "Dec 2009" folder.
5) Taxonomies are hierarchical organizations of content tags that are centrally managed. They provide a set of pre-defined tags used by content owners which greatly aids consistent tagging. I'll talk more about this later since it is a central feature of the Managed Metadata feature.
6) Tags can have synonyms that are automatically searched.
7) In addition to formal taxonomies, folksonomies provide decentralized and unmanaged metadata tagging. Folksonomies are not centrally defined or managed, but users freely add tags to content or reuse tags that other users have submitted.
8) Managed Metadata is centrally administrated which means that metadata configurations can be shared across site collections and farms. That means that content types no longer need to be migrated from test to production. Instead, content types are centrally stored and managed. Any site can subscribe to a particular content type. This service permits organizations to establish and control enterprise taxonomy for all SharePoint content.

Other Notable Features of Managed Metadata

1) Taxonomies can be imported and exported so that they can easily be shared between farms.
2) Administrative tools allow special users to create taxonomies
3) Administrators can control how tags from taxonomy are applied. In other words, taxonomy terms can have policies applied to them so as, for instance, to allow only certain terms to be used in particular sites.

Other notable enhancements of the search:

1) Click on a link in the search results and you jump directly to the Office application for that document.
2) "Did you mean" suggestions appear to help with misspelled queries.
3) Acronyms are expanded - search for ECM and get everything that mentions "Enterprise Content Management"
4) On Windows 7, search results can be integrated with desktop search.

I'll provide more detail later in later search sessions. )

Document Library Enhancements

1) Can drag documents directly into the library from the desktop or file system.
2) Document Sets: The capability to group multiple documents into a single entity. A Document Set is a content type that contains child content types used by the documents that comprise the set. The set can be versioned as a whole and downloaded as a single zip file. Workflows can be applied to the document set so that all the documents can be routed as a unit for an approval process for example. When we view the document set in a document library, it opens like a folder but displays a welcome page which displays the metadata and the list of documents (and can be easily customized).

New eDiscovery Features

1) Holds

- User can apply holds directly from the library
- Users can copy items to another location and have policies applied automatically to those items at the location where they are moved to
- Detailed reports on holds are available

Records Management: This is an area that has been almost completely revamped from MOSS.

1) Users can declare documents as to be records through a right click menu option.
2) Full auditability is provided around record declaration
a. Compliance details available for user inspection
b. Records management now has a hierarchical file plan with the ability to apply policies at multiple levels in a similar way to what we do currently with security groups
3) Information policies can now be applied at specific levels.
4) We can now implement a multi-phased disposition schedule
5) Content Organizer: This feature adds routing abilities that extend, enhance, and makes more broadly available the routing engine used in the Records Center site template from SharePoint 2007. It allows you to automatically route documents to different libraries and folders within those libraries.

It provides other abilities as well:

Making sure that folder items are limited to 5000. When item number 5001 is added to the library, the Content Organizer can automatically create a new folder and put the document in that folder and rename the folder according to rules you create.
Metadata can be applied to files based on property-based conditions, for instance, here is an example rule that could be applied automatically when a file is uploaded:
deal-size > 1M, so set the priority field to Critical. These rules can be applied at the site collection, site, or library level.

6) SharePoint Designer: Interface and functionality have significantly changed. It can now be used to design workflows visually. It displays visual representation of tasks completed.

7) File Server Coexistence: Coexistence refers to the strategy for sharing files between SharePoint and network drives. The old controversy of file Shares vs. SharePoint is becoming resolved as the platforms are being brought together. This strategy is enabled by the following features:
a. Metadata can now be applied to files in a file share
b. Can upload to SharePoint with the file metadata
i. You can move a file and leave a link to the file in the library. This permalink facility is enabled by the new Document ID that each document is assigned and acts as the permanent identifier.
ii. You can filter what's moved into SharePoint from the file system by property definitions working from the file metadata. For instance, what if you want to transfer files to SharePoint, but wanted to ensure that no sensitive files were not placed in SharePoint, you could make a rule that states that all the files will be checked for a Social Security Number. This can be detected whether the SSN is in a text file, a tif file or even if it has been redacted.

8) Rich media management
a. When retrieving media files they are automatically categorized by type and broken down into a tree view with the content types
i. Audio
ii. Video
iii. Image
9) Metadata can be assigned to media files through selection from taxonomy

SharePoint 2010 Enterprise Search Overview

There are three major SharePoint 2010 search products

1) Search Server Express
2) SharePoint Server 2010 Search
3) FAST Search Server

There are parallel tracks for the SharePoint Search and the FAST search. These are two separate options with distinct differences. The following are features of the new SharePoint Search:

1) Auto complete functionality is built in to the search box. As soon as you start typing, choices appear in the same way you see on Google.
2) The results page is not just a list of results as in 2007. On the left side of the page is the search refinement area where metadata extracted from the items to create a set of facets that categorize the results.
3) These facets can also be represented as a tree view so that users can drill into facets and sub facets.
4) Users can navigate over the result set by result type: web pages, file types,
Author, Modified Date, Sites are available facets out of the box.

Other notable new search features:

1) Federation to the desktop.
2) Social behavior improves search results relevance by processing the click-through rate and the document rating as part of the relevance ranking.
3) Wildcard search now included OOB.

People Search

1) You can now search for people even if you remember only part of the name.
2) People search results can be refined by Job Type: i.e. Salesperson or refined by Expertise, Past Projects (e.g. Market Analysis, Lean Project Management), Interests, Schools. Any custom metadata for the profile can be used, including My Sites data including Colleagues. Search results allow users to determine who created the most content of a certain type, an important point in evaluating the value of the content.
4) There is a link to Update my keywords which allows you to determine what are the keyword searches that lead to your profile. This helps determine what keywords should you put in your profile so that people can find you.
5) Users can scroll through PP slides directly form the search results screen.

Visual best bets

1) Federated results, including federation to the desk top.
2) Previews of the some document types built in to the search results.

Search Infrastructure

Significant changes have been made to the search services. The basic thrust is to componentize the indexing infrastructure, removed bottlenecks due to the single mega-index structure used in 2007. In particular, the index has been divided into partitions. "An index partition represents a portion of the entire index, and therefore the index is the aggregation of all index partitions. Partitioning the index allows different portions of the index to be spread across query servers. Administrators decide on the number and configuration of each of the partitions."

1) Unlike the current version, Multiple crawlers can be used to crawl different content simultaneously.
2) Search performance is much improved with subsecond latency scaling to 100M documents.

Connectors

1) Though they didn't mention an OOB connector to Documentum, they emphasized that it was easy to connect to Documentum using the new connector architecture. I'm going to investigate this as much as possible over the next few days. The long hunt for the best solution to Documentum and SharePoint integration may be closer.
2) SharePoint now can connect to any source that supports OpenSearch interface
3) Asset Management Systems were mentioned as system

Manageability

1) Consolidated dashboard.
a. Can access items directly in the crawl log
b. Query latency statistics help refine response time
c. The number of queries
d. Search queries and the number of results
2) Advanced Content Processing
a. Custom extraction of content
b. Crawl properties
c. Managed properties can be made into sort, query, or refiner properties by checkbox

3) Search configurations
a. Web Parts are now open can be extended
b. The search platform can support the fusion of structured and unstructured content
c. The presenter showed a project application that allowed users to add people and documents directly into their workspace from the search results screen and they were able to build presentations directly in the workspace from search results.
e. The search platform by itself appears to be an outstanding application platform

SharePoint 2010 First Impressions

First, the high level. The first SharePoint 2010 public beta will not be released until sometime in November, and frankly, after seeing some of the application crashes today, I'm glad. In one memorable episode during the Enterprise Search Overview, the presenter went through a long, convoluted process to show how "easy" it was to create a Visual Best Bet, only to have the browser crash. On the bright side, however, there are a lot of new features that directly address many of the key pain points and opportunities that we have met over the past 3 years of SharePoint implementation. In particular:

1) A filterable metadata picker that allows users direct access to enterprise taxonomies. It appears that it will be much easier to create a filtered list of metadata values (filtered by site or library) such as we have long thought about doing programmatically. In fact, the new version may have this functionality built in.
2) The new version has built-in media streaming, which will take care of the needs we indentified during the Entergy proposal.
3) The new search has built-in faceted searching similar to Coveo, but tightly integrated into the SharePoint interface.
4) A document rating system is built-in to document libraries and the ratings become part of the relevance ranking of search results.
5) The search platform allows integration between structured and unstructured content.
6) Visio services are directly integrated into SharePoint, which allows Visio diagrams to become one option for an interface into SharePoint content. I think this will directly address an idea Jeff and I have been batting around for SharePoint interfaces. It looks like they were one step ahead of us - at least in providing the tools. Here is a description of one of the sessions dealing with this: "This session will drill down into ways that you can create and integrate Visio visualizations inside of SharePoint using Visio Services. Learn how to use Visio Services to complement existing SharePoint Business Intelligence applications in creative ways. We will also cover all of the complimentary technologies on the server such as the JavaScript API and custom data sources."

Those are some of the first day highlights. I'm going into detail in the following three blog posts for each of the sessions I attended today.