Sitecore SPEAK: ProgressBar Event Trigger

In Sitecore SPEAK development, the ProgressBar control displays a horizontal bar. It can be specified a value between 0 and 100, and the control displays part of the bar in color to indicate the level of progress.

How to update the progress? This control has a property “UpdateInterval” which specify an update interval in milliseconds. The ProgressBar control raises an event at the end of each of these intervals. The event is called intervalCompleted + the ID of the control. custom code can listen for this event in page code. For example the control ID is “ProgressBar1”. Here is the code.

initialized: function () {
 this.ProgressBar1.on("intervalCompleted:ProgressBar1", this.intervalCompletedprgBar, this);
 },

intervalCompletedprgBar: function ()
 {
 var currBar = this.ProgressBar1.attributes.value;
 this.ProgressBar1.set("value", currBar+(100 - currBar) / 5);
 }

The event is set during code initialized and supposed to be triggered at the end of each of intervals. But the event has never been triggered. After contacting Sitecore Support several rounds, at the end it was reported and registered as a bug in Sitecore bug tracking system. To track the future status of this bug report, use the reference number 198936.

Is there any workaround? Yes. Here is a solution.

This is the Sitecore outbox ProgressBar JavaScript: \sitecore\shell\client\Business Component Library\Layouts\Renderings\Common\ProgressBars\ProgressBar.js

setupTimer: function () {      
         var updateInterval = this.model.get("updateInterval");      
         clearInterval(this.timer);      
         if (updateInterval <= 0) {       
               return;      
         }      
         var id = this.$el.attr("data-sc-id");      
         this.timer = setInterval(function () {        
              _sc.trigger("intervalCompleted:" + id);     
         }, updateInterval);
},

Highlight line code suppose to trigger ProgressBar event, but it doesn’t. The workaround is, replace it with calling progress bar update function directly. The only thing need to be aware is future Sitecore upgrade.

setupTimer: function () {      
         var updateInterval = this.model.get("updateInterval");      
         clearInterval(this.timer);      
         if (updateInterval <= 0) {       
               return;      
         }      
         var id = this.$el.attr("data-sc-id");      
         this.timer = setInterval(function () {        
              //_sc.trigger("intervalCompleted:" + id); 
              _sc.app.intervalCompletedprgBar(); 
         }, updateInterval);
},

 

Advertisements

Sitecore Request Validation Configuration

This was an interesting experience and definitely I’d like to share the tricks learned during this website developed in Sitecore for one of our clients.

This site was migrated from an existing one. Everything was fine in UAT/staging and ready for release.  But when switching the DNS, instead of seeing beautiful home page, there was ugly yellow/red .net error message showing up like:

HttpRequestValidationException: A potentially dangerous Request.Cookies value was detected from the client… 

Immediately, questions came up, what’s that? why?

Since the message mentioned cookies, then I checked browser and found there was a cookie created by previous site(migrated from) when I browsed it. The cookie value contained html tags like “<b>”, “<br>”, even they were harmless. Now it was clear that the new site page request was blocked by .Net Http Runtime request validation.

Sitecore is a platform built upon .Net framework. Request validation is a feature in ASP.NET that examines an HTTP request and determines whether it contains potentially dangerous content. In this context, potentially dangerous content is any HTML markup or JavaScript code in the body, header, query string, or cookies of the request. ASP.NET performs this check because markup or code in the URL query string, cookies, or posted form values might have been added for malicious purposes.

Based on Microsoft HttpRuntimeSection.RequestValidationMode Property Document, here are available settings:

• 4.5(the default). In this mode, values are lazily loaded, that is, they are not read until they are requested.
• 4.0. The HttpRequest object internally sets a flag that indicates that request validation should be triggered whenever any HTTP request data is accessed. This guarantees that the request validation is triggered before data such as cookies and URLs are accessed during the request. The request validation settings of the element (if any) in the configuration file or of the directive in an individual page are ignored.
• 2.0. Request validation is enabled only for pages, not for all HTTP requests. In addition, the request validation settings of the element (if any) in the configuration file or of the directive in an individual page are used to determine which page requests to validate.

When I checked the installed Sitecore instance web.config, by default the “requestValidationMode” was set to 4.0, like below:

<httpRuntime targetFramework="4.5.2" maxRequestLength="512000" executionTimeout="3600" enableKernelOutputCache="false" relaxedUrlToFileSystemMapping="true" requestValidationMode="4.0" enableVersionHeader="false" />

After I changed it to 4.5 which is lazy loading, the site displayed fine. Then we went through all code to make sure the request validation were handled at page level as requested.

 

Sitecore DevOps 1-2-3

DevOps is a hot topic in today’s software development along with Agile methodology. What does DevOps mean?

The term DevOps is commonly considered a combination of the concepts of development and operations. It is used in IT to refer to roles or processes that bridge various department – usually development and operations teams – to achieve a certain project management philosophy that involves more efficiency in communications between development teams and other parts of a greater business or organization.

– Techopidia

From technical perspective, the key points of DevOps are continuous integration and auto-deployment. In real world, we always experience functionalities and features in different stages of development life cycle cross all environments: Dev, QA, Staging, UAT, Production. Sitecore even brings additional complexity with various instances CM, CD, Processing, Reporting, etc.

Below is an overview of Sitecore development life cycle and operation procedure with different stages audience and roles.

How can DevOps orchestrate Sitecore implementation from various departments to deploy code and items to different environments/instances? Not using Sitecore Ship, Unicorn, Gulp, Grunt, Git, TeamCity, you name it, I came up a simple way to create Sitecore DevOps CI/CD procedure with popular .Net development IDE and tools:

• Visual Studio
• Team Foundation Server
• SlowCheetah
• TDS

First, create code solution based on Sitecore Helix principle and Habitat architecture approach.

This is fundamental to entire DevOps. (Ref: http://helix.sitecore.net/). Because Sitecore platform multitenancy approach results all client websites, applications and components built within Sitecore developed in single code base. So the Modular Architecture is Sitecore recommended best practice. It moves away from the traditional 3 tiered architecture (UI, business logic, data). It breaks the solution into smaller independent and manageable parts. Each module is self-contained with business logic, UI and TDS Sitecore items. It can be built, packed and shipped independently without interfering other modules causing massive regression test. The module should only depend upon modules that are more stable than it.

Second, create build definitions with branch change sets and transformations.

This will differentiate feature stages and environment settings during continue integration and continue deployment. By using TFS merge and branch function, we can move features by checked-in change sets, see below.

In Visual Studio, create configuration profiles for each Sitecore environment: Dev; StgCM; StgCd1; StgCd2; ProdCM; ProdCd1; ProdCd2, etc. Add SlowCheetah extension to Visual Studio.  It will transform all custom Sitecore configuration files at build time based on profile.

Third, use TDS Sitecore Package Deployer to automatically process Sitecore update packages.

• In Visual Studio TDS project properties, enable “Generate package during build” according to the configuration profile.
• Create a folder “SitecorePackageDeployer” under “Data”.
• Install Sitecore Package Deployer. It includes a scheduled task run by the Sitecore task agent. This task runs one per minute and looks for packages in the folder created above
• In TFS build definition and release task, make sure the generated web code published to Sitecore “Website” folder and update packages published to Sitecore “Data\SitecorePackageDeployer” folder.

 

Build a Sitecore Module With External Big Data through Solr

Sitecore stores and manages content through Sitecore items. They are actually data saved in relational SQL Sever database. Normally the content managed in Sitecore are better created as Sitecore items. But in today’s enterprise ecosystem, many external data cannot be created into Sitecore because:

• There are existing applications managing the external data, no need to create save function in Sitecore. For example, PIM(Product Information/Inventory Management)
• External data is huge and ETL might take long time to sync data to Sitecore. For example product inventory data in retail business by SKU, Home Depot sells 300K+, BestBuy sells 725K+; in travel booking system by vacation package, Sunwing sells 10M+
• External data changes very frequently. For example dynamic price of flight ticket and hotel room, real time merchandise inventory.

So the right approach is,

• Integrate with external data, no ETL to Sitecore
• Conduct real time search against external data through Solr

Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene. Installing Solr is easy, see Learn more about Solr.  Here is an example with key points to create a core in Solr and configure index to a table in SQL Server.

• solrconfig:

  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar" /> 
  
  <lib dir="${solr.install.dir:../../../..}/dist/" regex="sqljdbc42\.jar" />

• managed-schema:

  <field name="ProductID" type="long" indexed="true" required="false" stored="true"/>
  
  <field name="Name" type="string" docValues="true" indexed="true" stored="true"/>
  
  <field name="ProductNumber" type="string" docValues="true" indexed="true" stored="true"/>

• db-data-config:

  <dataSource driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" url="jdbc:sqlserver://localhost:8983;databaseName=Northwind" user="sa" password="xxxxxx" />
  
  <entity name="item"  query="select * from Products"  deltaQuery="select ProductID from Products and updateDate > '${dataimporter.last_index_time}‘”  deltaImportQuery=“” deletedPkQuery=“” transformer="RegexTransformer,DateFormatTransformer,TemplateTransformer"> </entity>

SolrNet is an Apache Solr client for .NET to interact with the Solr search engine. Sitecore installation package officially supports Lucene for native and Solr for distributed indexing. Since Sitecore 8.2 the SolrNet is included in package which means SolrNet has been fully test by Sitecore. Programming with SolrNet is easy. Here is an example with steps:

• Initialize:

   

  Startup.Init<Bike>("http://localhost:8983/solr/Bike");    
  
  ISolrOperations<Bike> solr =  ServiceLocator.Current.GetInstance<isolroperations<Bike>>();

• Query in SolrNet:

  var qSize = new SolrQueryByRange<int>("Size", 40, 52);
  
  var qSubCat = new SolrQueryByField("ProductSubcategory", "Mountain Bikes");
  
  var qColor = new SolrQueryInList("Color", "Silver", "Black");
  
  SolrQueryResults<Bike> bikes = solr.Query(qSize && qSubCat && qColor);

• Query Converted to Solr REST Api:

  http://localhost:8983/solr/Bike/select?fq=Color:((Silver)%20OR%20(Black))&fq=ProductSubcategory:%20%22Mountain%20Bikes%22&fq=Size:[40%20TO%2052]&indent=on&q=*:*&wt=json

A trick to set up 301 url redirect in Sitecore

Recently I am working on a task to redirect whole bunch of page urls to a new location. I want to preserve all of the links from search engines. So I need to make sure that the old links give a 301 (Moved Permanently) response back. This will allow the web crawlers to update their databases with the new links as well as keep the old links working in the meantime.

I downloaded the Sitecore 301 redirect module from marketplace. It comes up with three types redirect setting:

• Exact Match Redirects
• Regular Expression Redirects
• Rules based Redirects

It sounds very straightforward. I started to set a redirect with type “Exact Match Redirects”. For example:

http://somesite.com/en/oldpage.aspx –> Sitecore/Content/somesite/newpage.

But no matter how I tried, the redirect just didn’t happen. What’s wrong?

After reading the document and tried several more different scenarios, I found how the redirect module works. Sitecore allows customizing the rendering pipeline which is exactly the redirect module does. This module kicks in right after Sitecore tries to resolve the item for the request. If Sitecore resolves the item, this module doesn’t do anything. Otherwise, it runs through the rules in the above 3 matching order.

One important thing I found is, Sitecore’s language processor resolves the language before rendering happens by retrieving the language flag in url. That means when this redirect modules kicks in to compare the match url, the language has already been solved and the flag in url is removed. For example above old url becomes http://somesite.com/oldpage.aspx. That’s why my previous redirect settings failed. After I removed the language flag from the url, it succeeded.

Another tip to make the redirect work is, make sure the old link is dead/disabled.

An Issue found in Sitecore User Profile Storage

Recently one of our content authors started to get error like this very frequently.

“Length of LOB data (65608) to be replicated exceeds configured maximum 65536. Use the stored procedure sp_configure to increase the configured maximum value for max text repl size option, which defaults to 65536. A configured value of -1 indicates no limit, other that the limit imposed by the data type….”

It only happens to her account, not others. She manages the site content for about one year. From the error message itself, it is very clear that the SQL Server max text replication need to be increased from default 65536 to a bigger size. I know this SQL server configuration change will fix the issue. But the strange part is what cause the issue and why only happen to specific user account?

I start to trace from SQL Server which Sitecore data is stored. The default setting of “Max Text Replication Size” in SQL Server is 65536 bytes = 0.0625MB. This configuration specifies the maximum size of data that can be added to a replicated column in a single INSERT, UPDATE, WRITETEXT or UPDATETEXT statement. This applies to data type text, ntext, varchar (max), nvarchar (max), varbinary (max), xml, and image.

Sitecore has 3 major SQL databases Core, Master and Web. Since this error happens on Sitecore console application to specific user account, I checked the Core database first to review those table columns with data type in text, ntext, varchar (max), nvarchar (max), varbinary (max), xml, and image. One table come to my eyes, it is “aspnet_Profile”.

By default, Sitecore uses ASP.Net Membership Provider to manage authentication and user profile. These tables are in the Core database. The standard Microsoft implementation of the SqlProfileProvider (which is used in Sitecore by default) stores the user profile information in the “aspnet_Profile” table.

SqlProfileProvider persists profile properties in three fields of the “aspnet_Profile” table: PropertyNames, PropertyValuesString, and PropertyValuesBinary.

The following is a synopsis of what’s stored in each field:  PropertyNames holds a string value containing information about the profile property values present in the PropertyValuesString and PropertyValuesBinary fields. The string holds a colon-delimited list of items; each item denotes one property value, and it is encoded in the following format: Name:B|S:StartPos:Length

PropertyValuesBinary is the field I’d like to check further to determine if it grows wild as it is image data type. Then I use following query to watch the data change in the “aspnet_Profile” table.

SELECT [UserId],[PropertyNames],[PropertyValuesString],
[PropertyValuesBinary],[LastUpdatedDate]  
FROM [dbo].[aspnet_Profile] order by LastUpdatedDate desc

The experiment I did included following activities:

• Create page and content
• Modify the page content in content/experience editors
• Submit page content for approval as workflow required
• Publish page content

Then I found the size of PropertyValuesBinary only increase when executing workflow command from the Sitecore Content Editor, Experience Editor or Workbox on single item with comment, see below. Every workflow command execution increased about 159 bytes.

Now it explains me the reason why some author accounts started experiencing the error. After websites live for a length of period depends how active authors’ actions in workflow, such as submitting draft, approving, rejecting, etc. Once the data size of PropertyValuesBinary exceeds 65536, the error will show up. I don’t know why Sitecore saves these type of information in “aspnet_Profile” table as binary data, and don’t find how I can reduce or remove them from application console. So here are two solutions: run the following script in Core database to increase the limitation, or create a new account for the author to replace her previous one.

exec sp_configure ‘max text repl size’, 2147483647
Go
RECONFIGURE
GO

Make ReactJS work in Sitecore Environment

 

In today’s front end landscape, the JavaScript technology is playing a very important role in UI and UX.   It is changing very quick and new technology is coming out every month. Currently you probably know the two most popular JS libraries: Angular and React. In Sitecore development, JQuery is very popular adopted pretty much from beginning as part of .Net development. So question is, can we have most modern JS libraries like Angular and React work with Sitecore?

Yes. Before going to the detail, I’d like to simply compare the difference between Angular and React in a high level. In my opinion, Angular is more like a framework. To take it’s advantage, the entire system is better designed to work in Angular way. But React is more like a design pattern. It solves front-end view only and system can be designed in its own way. So to work with Sitecore, I prefer React personally, especially with ReactJS.Net.

The best practice of Sitecore is component-based design or modular architecture. In a Sitecore .Net MVC environment, the base UI element is rendering which is a group of data model, business controller, logic action and view. React is component-based too. It builds encapsulated components that manage their own state, then compose them to make complex UIs. React components implement a render() method that takes input data and returns what to display. You can see both Sitecore and React use the similar term to describe UI element “Rendering” or “Render”.

ReactJS.Net makes .Net developers to work with React and JSX much easier. Thanks Babel to transform JSX to JavaScript and cached server side automatically on-the-fly. It supports ECMAScript 6 features, Server-side rendering and ASP.Net bundling and minification.

Here is a sample I applied React in Sitecore. There is an application which splits traffic flow randomly by percentage. The visitors IP information are stored in database. The requirement is, streaming the traffic split information at real time and display in a grid with additional information by looking up MaxMind Geo data service. It has a drop down search box for filtering the results.

• Use NuGet to install the ReactJS.NET (MVC 4 and 5) package.
• Create MVC controller, action and view
• Create React component in JSX with proper name space for self-contained scope to avoid conflict variables defined globally.
  • main.jsx

    var Traffics = trafficMonitorApp.Components.Traffics
    
    ReactDOM.render(
     <Traffics url="/Home/Traffics" pollInterval={5000} />,
     document.getElementById('content')
    );

    

• Similar to MVC view and partial view, split React component to structured renders for better management.

• Load data from server-side action initially or retrieve from Web API through AJAX call at real time.
• Bundle the JSX files in BundleConfig.cs.
• Invoke the JSX bundle from view.

  @Scripts.Render("~/bundles/main")

• Define the rendering in Sitecore as normal controller or view rendering.

Here is the completed component: