Sitecore 9 Installation – Standalone

Here is a step by step guide of Sitecore 9 on-premise installation as a standalone instance. It is mostly for local workstation and integrated development environment.

Key points of this installation:

  • Topology: XP0

no-scaling-simple

  • Solr is installed by default. It serves both content search and xConnect services. It requires secure access.
  • Sitecore 9 is installed by SIF(Sitecore Installation Framework) PowerShell scripts.

Prerequisites:

  • Windows 8.1, 10, Server 2012 R2
  • Net Framework 4.6.2 or 4.7
  • SQL Server 2016 or above
  • IIS 8.5 or 10
  • Java Runtime Environment 1.8+(JRE)
  • PowerShell 5.1+
  • Web Platform Installer 5.0
  • Web Administration Module (IIS)
  • Web Deploy 3.6 for Hosting Servers
  • URL Rewrite 2+
  • MS SQL Server Data-Tier Application Framework (17.1)
  • MS SQL Server Transact-SQL ScriptDom

Download and Collect Installation Related Files:

  • Sitecore License.xml
  • Download Sitecore 9.0.2 rev. 180604 (WDP XP0 packages).zip from https://dev.sitecore.net/
  • Extract Sitecore 9.0.2 rev. 180604 (WDP XP0 packages).zip to an install folder. For example, C:\Sitecore\sc902.
  • Extract XP0 Configuration Files 9.0.2 rev. 180604.zip from previous step to C:\Sitecore\sc902.
  • Download Solr 6.6.3 from http://archive.apache.org/dist/lucene/solr/6.6.3/ and extract to C:\Sitecore\solr-6.6.3
  • Download NSSM from https://nssm.cc/download and extract to C:\Sitecore\nssm
  • Here are the files collected.

SIF02

Install Solr:

SIF04

  • Start Solr – run following script in Windows PowerShell as Administrator, then check solr instance at http://localhost:8983/solr
cd C:\Sitecore\solr-6.6.3\bin

.\solr start -p 8983

SIF05

  • Stop Solr – run following script in Windows PowerShell as Administrator
.\solr stop -p 8983
  • Set up Solr as service – run below script in Command Prompt as Administrator, then fill the installer as the picture below. Once done, verify the service running from Service Manager.
cd C:\Sitecore\nssm\win64

nssm install solr6

SIF06

 

param(
[string]$KeystoreFile = 'solr-ssl.keystore.jks',
[string]$KeystorePassword = 'secret',
[string]$SolrDomain = 'localhost',
[switch]$Clobber
)

$ErrorActionPreference = 'Stop'

### PARAM VALIDATION
if($KeystorePassword -ne 'secret') {
Write-Error 'The keystore password must be "secret", because Solr apparently ignores the parameter'
}

if((Test-Path $KeystoreFile)) {
if($Clobber) {
Write-Host "Removing $KeystoreFile..."
Remove-Item $KeystoreFile
} else {
$KeystorePath = Resolve-Path $KeystoreFile
Write-Error "Keystore file $KeystorePath already existed. To regenerate it, pass -Clobber."
}
}

$P12Path = [IO.Path]::ChangeExtension($KeystoreFile, 'p12')
if((Test-Path $P12Path)) {
if($Clobber) {
Write-Host "Removing $P12Path..."
Remove-Item $P12Path
} else {
$P12Path = Resolve-Path $P12Path
Write-Error "Keystore file $P12Path already existed. To regenerate it, pass -Clobber."
}
}

try {
$keytool = (Get-Command 'C:\Program Files\Java\jre1.8.0_172\bin\keytool.exe').Source
} catch {
$keytool = Read-Host "keytool.exe not on path. Enter path to keytool (found in JRE bin folder)"

if([string]::IsNullOrEmpty($keytool) -or -not (Test-Path $keytool)) {
Write-Error "Keytool path was invalid."
}
}

### DOING STUFF

Write-Host ''
Write-Host 'Generating JKS keystore...'
& $keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass $KeystorePassword -storepass $KeystorePassword -validity 9999 -keystore $KeystoreFile -ext SAN=DNS:$SolrDomain,IP:127.0.0.1 -dname "CN=$SolrDomain, OU=Organizational Unit, O=Organization, L=Location, ST=State, C=Country"

Write-Host ''
Write-Host 'Generating .p12 to import to Windows...'
& $keytool -importkeystore -srckeystore $KeystoreFile -destkeystore $P12Path -srcstoretype jks -deststoretype pkcs12 -srcstorepass $KeystorePassword -deststorepass $KeystorePassword

Write-Host ''
Write-Host 'Trusting generated SSL certificate...'
$secureStringKeystorePassword = ConvertTo-SecureString -String $KeystorePassword -Force -AsPlainText
$root = Import-PfxCertificate -FilePath $P12Path -Password $secureStringKeystorePassword -CertStoreLocation Cert:\LocalMachine\Root
Write-Host 'SSL certificate is now locally trusted. (added as root CA)'

Write-Host ''
Write-Host '########## NEXT STEPS ##########' -ForegroundColor Green
Write-Host ''
Write-Host '1. Copy your keystore to $SOLR_HOME\server\etc (MUST be here)' -ForegroundColor Green

if(-not $KeystoreFile.EndsWith('solr-ssl.keystore.jks')) {
Write-Warning 'Your keystore file is not named "solr-ssl.keystore.jks"'
Write-Warning 'Solr requires this exact name, so make sure to rename it before use.'
}

$KeystorePath = Resolve-Path $KeystoreFile
Write-Host ''
Write-Host '2. Add the following lines to your solr.in.cmd:' -ForegroundColor Green
Write-Host ''
Write-Host "set SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.jks" -ForegroundColor Yellow
Write-Host "set SOLR_SSL_KEY_STORE_PASSWORD=$KeystorePassword" -ForegroundColor Yellow
Write-Host "set SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.jks" -ForegroundColor Yellow
Write-Host "set SOLR_SSL_TRUST_STORE_PASSWORD=$KeystorePassword" -ForegroundColor Yellow
Write-Host ''
Write-Host 'Done!'
  • Get it to your local and modify the “keytool.exe” path inside if need(line 37), see below.

SIF07

    • Run follwing command in Windows PowerShell as Administrator.
cd C:\Sitecore\sc902

.\solrssl.ps1 -KeystoreFile C:\Sitecore\solr-6.6.3\server\etc\solr-ssl.keystore.jks
    • Enable a section in file “C:\Sitecore\solr-6.6.3\bin\solr.in.cmd” (line 94-97), see below.SIF08
    • Re-start the Solr service and check instance at https://localhost:8983/solr.

Install and Configure SQL Server:

  • The SQL Server installtion is pretty much same as Sitecore previous version installation.
  • When creating a SQL Server DB user, make sure it has “sysadmin” role. The login Id and password will be used in Sitecore installation script next.
  • Enable Contained Database Authentication – run following sql script in SQL Management Studio
sp_configure 'contained database authentication', 1; GO RECONFIGURE; GO

Install Sitecore with SIF:

Sitecore Installation Framework(SIF) is a PowerShell module for handling the installation and configuration of Sitecore instances. It will install and configure Sitecore eco-system.

Run following script in Windows PowerShell as Administrator and type “Y” to continue if need.

  • Register Sitecore Repository
Register-PSRepository -Name SitecoreRepo -SourceLocation https://sitecore.myget.org/F/sc-powershell/api/v2
  • Install Framework module
Install-Module SitecoreInstallFramework
  • Install Fundamentals module
Install-Module SitecoreFundamentals
  • Import above modules
Import-Module SitecoreFundamentals

Import-Module SitecoreInstallFramework
  • Configure the PowerShell script below and save as “InstallSitecore9.ps1”. Before running, check and update parameters values to you environment.
#define parameters 
$prefix = "sc90"
$PSScriptRoot = "C:\sitecore\sc902"
$XConnectCollectionService = "$prefix.xconnect"
$sitecoreSiteName = "$prefix.local"
$SolrUrl = "https://localhost:8983/solr"
$SolrRoot = "C:\sitecore\solr-6.6.3"
$SolrService = "solr6"
$SqlServer = ".\SQLEXPRESS"
$SqlAdminUser = "sc9user" # replace it with your local SQL Server installation
$SqlAdminPassword = "Test12345" # replace it with your local SQL Server installation

#install client certificate for xconnect 
$certParams = 
@{ 
Path = "$PSScriptRoot\xconnect-createcert.json" 
CertificateName = "$prefix.xconnect_client" 
} 
Install-SitecoreConfiguration @certParams -Verbose

#install solr cores for xdb 
$solrParams = 
@{
Path = "$PSScriptRoot\xconnect-solr.json" 
SolrUrl = $SolrUrl 
SolrRoot = $SolrRoot 
SolrService = $SolrService 
CorePrefix = $prefix 
} 
Install-SitecoreConfiguration @solrParams -Verbose

#deploy xconnect instance 
$xconnectParams = 
@{
Path = "$PSScriptRoot\xconnect-xp0.json" 
Package = "$PSScriptRoot\Sitecore 9.0.2 rev. 180604 (OnPrem)_xp0xconnect.scwdp.zip"
LicenseFile = "$PSScriptRoot\license.xml"
Sitename = $XConnectCollectionService 
XConnectCert = $certParams.CertificateName 
SqlDbPrefix = $prefix 
SqlServer = $SqlServer 
SqlAdminUser = $SqlAdminUser
SqlAdminPassword = $SqlAdminPassword
SolrCorePrefix = $prefix
SolrURL = $SolrUrl 
} 
Install-SitecoreConfiguration @xconnectParams -Verbose

#install solr cores for sitecore 
$solrParams = 
@{
Path = "$PSScriptRoot\sitecore-solr.json"
SolrUrl = $SolrUrl
SolrRoot = $SolrRoot
SolrService = $SolrService 
CorePrefix = $prefix 
} 
Install-SitecoreConfiguration @solrParams -Verbose

#install sitecore instance 
$sitecoreParams = 
@{ 
Path = "$PSScriptRoot\sitecore-XP0.json"
Package = "$PSScriptRoot\Sitecore 9.0.2 rev. 180604 (OnPrem)_single.scwdp.zip" 
LicenseFile = "$PSScriptRoot\license.xml"
SqlDbPrefix = $prefix 
SqlServer = $SqlServer 
SqlAdminUser = $SqlAdminUser 
SqlAdminPassword = $SqlAdminPassword 
SolrCorePrefix = $prefix 
SolrUrl = $SolrUrl 
XConnectCert = $certParams.CertificateName 
Sitename = $sitecoreSiteName 
XConnectCollectionService = "https://$XConnectCollectionService" 
} 
Install-SitecoreConfiguration @sitecoreParams -Verbose
  • Open the script file “InstallSitecore9.ps1” in Windows PowerShell ISE to execute or copy the scripts to run in Windows PowerShell as Administrator.
  • Confirm Sitecore 9 installed successfully.

SIF10

Advertisements
Posted in Information Technology, Sitecore | Tagged , , , , , | Leave a comment

Sitecore CMS-Only Mode and Submit Queue

Prior to version 9, Sitecore use MongoDb to store analytics data. If you don’t need analytics features and don’t want to maintain MongoDb,  you may like to configure Sitecore as CMS-only mode. Based on Sitecore document, it is pretty straightforward.

In Sitecore configuration file “Sitecore.Xdb.config”

<setting name="Xdb.Enabled" value="false" />

It prevents the data from being saved to the database. Now it is safe to remove MongoDb connection strings.

But the tracking is still happening. there is a Sitecore ‘Submit Queue’ that flushes captured data to the server disk (the ‘Data\Submit Queue’ folder by default). It expects MongoDb comes back online, a job process submits the data to MongoDb from the disk “Submit Queue” folder.

But this may cause performance issue to authoring server and even possible item publish error when server runs for a period. To solve this,

In file “Sitecore.Xdb.config”

<setting name="Xdb.Tracking.Enabled" value="false" />

In file “Sitecore.Analytics.Tracking.config”.

<setting name="Analytics.UseSubmitQueue" value="false" />
Posted in Uncategorized | 2 Comments

Sitecore SPEAK: ProgressBar Event Trigger

In Sitecore SPEAK development, the ProgressBar control displays a horizontal bar. It can be specified a value between 0 and 100, and the control displays part of the bar in color to indicate the level of progress.

How to update the progress? This control has a property “UpdateInterval” which specify an update interval in milliseconds. The ProgressBar control raises an event at the end of each of these intervals. The event is called intervalCompleted + the ID of the control. custom code can listen for this event in page code. For example the control ID is “ProgressBar1”. Here is the code.

initialized: function () {
 this.ProgressBar1.on("intervalCompleted:ProgressBar1", this.intervalCompletedprgBar, this);
 },

intervalCompletedprgBar: function ()
 {
 var currBar = this.ProgressBar1.attributes.value;
 this.ProgressBar1.set("value", currBar+(100 - currBar) / 5);
 }

The event is set during code initialized and supposed to be triggered at the end of each of intervals. But the event has never been triggered. After contacting Sitecore Support several rounds, at the end it was reported and registered as a bug in Sitecore bug tracking system. To track the future status of this bug report, use the reference number 198936.

Is there any workaround? Yes. Here is a solution.

This is the Sitecore outbox ProgressBar JavaScript: \sitecore\shell\client\Business Component Library\Layouts\Renderings\Common\ProgressBars\ProgressBar.js

setupTimer: function () {      
         var updateInterval = this.model.get("updateInterval");      
         clearInterval(this.timer);      
         if (updateInterval <= 0) {       
               return;      
         }      
         var id = this.$el.attr("data-sc-id");      
         this.timer = setInterval(function () {        
              _sc.trigger("intervalCompleted:" + id);     
         }, updateInterval);
},

Highlight line code suppose to trigger ProgressBar event, but it doesn’t. The workaround is, replace it with calling progress bar update function directly. The only thing need to be aware is future Sitecore upgrade.

setupTimer: function () {      
         var updateInterval = this.model.get("updateInterval");      
         clearInterval(this.timer);      
         if (updateInterval <= 0) {       
               return;      
         }      
         var id = this.$el.attr("data-sc-id");      
         this.timer = setInterval(function () {        
              //_sc.trigger("intervalCompleted:" + id); 
              _sc.app.intervalCompletedprgBar(); 
         }, updateInterval);
},

 

Posted in Uncategorized | Leave a comment

Sitecore Request Validation Configuration

This was an interesting experience and definitely I’d like to share the tricks learned during this website developed in Sitecore for one of our clients.

This site was migrated from an existing one. Everything was fine in UAT/staging and ready for release.  But when switching the DNS, instead of seeing beautiful home page, there was ugly yellow/red .net error message showing up like:

HttpRequestValidationException: A potentially dangerous Request.Cookies value was detected from the client… 

Immediately, questions came up, what’s that? why?

Since the message mentioned cookies, then I checked browser and found there was a cookie created by previous site(migrated from) when I browsed it. The cookie value contained html tags like “<b>”, “<br>”, even they were harmless. Now it was clear that the new site page request was blocked by .Net Http Runtime request validation.

Sitecore is a platform built upon .Net framework. Request validation is a feature in ASP.NET that examines an HTTP request and determines whether it contains potentially dangerous content. In this context, potentially dangerous content is any HTML markup or JavaScript code in the body, header, query string, or cookies of the request. ASP.NET performs this check because markup or code in the URL query string, cookies, or posted form values might have been added for malicious purposes.

Based on Microsoft HttpRuntimeSection.RequestValidationMode Property Document, here are available settings:

  • 4.5(the default). In this mode, values are lazily loaded, that is, they are not read until they are requested.
  • 4.0. The HttpRequest object internally sets a flag that indicates that request validation should be triggered whenever any HTTP request data is accessed. This guarantees that the request validation is triggered before data such as cookies and URLs are accessed during the request. The request validation settings of the element (if any) in the configuration file or of the directive in an individual page are ignored.
  • 2.0. Request validation is enabled only for pages, not for all HTTP requests. In addition, the request validation settings of the element (if any) in the configuration file or of the directive in an individual page are used to determine which page requests to validate.

When I checked the installed Sitecore instance web.config, by default the “requestValidationMode” was set to 4.0, like below:

<httpRuntime targetFramework="4.5.2" maxRequestLength="512000" executionTimeout="3600" enableKernelOutputCache="false" relaxedUrlToFileSystemMapping="true" requestValidationMode="4.0" enableVersionHeader="false" />

After I changed it to 4.5 which is lazy loading, the site displayed fine. Then we went through all code to make sure the request validation were handled at page level as requested.

 

Posted in Information Technology, Sitecore | Tagged , , , | Leave a comment

Sitecore DevOps 1-2-3

DevOps is a hot topic in today’s software development along with Agile methodology. What does DevOps mean?

The term DevOps is commonly considered a combination of the concepts of development and operations. It is used in IT to refer to roles or processes that bridge various department – usually development and operations teams – to achieve a certain project management philosophy that involves more efficiency in communications between development teams and other parts of a greater business or organization.

– Techopidia

From technical perspective, the key points of DevOps are continuous integration and auto-deployment. In real world, we always experience functionalities and features in different stages of development life cycle cross all environments: Dev, QA, Staging, UAT, Production. Sitecore even brings additional complexity with various instances CM, CD, Processing, Reporting, etc.

Below is an overview of Sitecore development life cycle and operation procedure with different stages audience and roles.

Sitecore Development and Publish Procedure

How can DevOps orchestrate Sitecore implementation from various departments to deploy code and items to different environments/instances? Not using Sitecore Ship, Unicorn, Gulp, Grunt, Git, TeamCity, you name it, I came up a simple way to create Sitecore DevOps CI/CD procedure with popular .Net development IDE and tools:

  • Visual Studio
  • Team Foundation Server
  • SlowCheetah
  • TDS

First, create code solution based on Sitecore Helix principle and Habitat architecture approach.

This is fundamental to entire DevOps. (Ref: http://helix.sitecore.net/). Because Sitecore platform multitenancy approach results all client websites, applications and components built within Sitecore developed in single code base. So the Modular Architecture is Sitecore recommended best practice. It moves away from the traditional 3 tiered architecture (UI, business logic, data). It breaks the solution into smaller independent and manageable parts. Each module is self-contained with business logic, UI and TDS Sitecore items. It can be built, packed and shipped independently without interfering other modules causing massive regression test. The module should only depend upon modules that are more stable than it.

Second, create build definitions with branch change sets and transformations.

This will differentiate feature stages and environment settings during continue integration and continue deployment. By using TFS merge and branch function, we can move features by checked-in change sets, see below.Sitecore_DevoOps2

In Visual Studio, create configuration profiles for each Sitecore environment: Dev; StgCM; StgCd1; StgCd2; ProdCM; ProdCd1; ProdCd2, etc. Add SlowCheetah extension to Visual Studio.  It will transform all custom Sitecore configuration files at build time based on profile.

Third, use TDS Sitecore Package Deployer to automatically process Sitecore update packages.

  • In Visual Studio TDS project properties, enable “Generate package during build” according to the configuration profile.
  • Create a folder “SitecorePackageDeployer” under “Data”.
  • Install Sitecore Package Deployer. It includes a scheduled task run by the Sitecore task agent. This task runs one per minute and looks for packages in the folder created above
  • In TFS build definition and release task, make sure the generated web code published to Sitecore “Website” folder and update packages published to Sitecore “Data\SitecorePackageDeployer” folder.

 

Posted in Sitecore | Tagged , , , , , , , | 2 Comments

Build a Sitecore Module With External Big Data through Solr

Sitecore stores and manages content through Sitecore items. They are actually data saved in relational SQL Sever database. Normally the content managed in Sitecore are better created as Sitecore items. But in today’s enterprise ecosystem, many external data cannot be created into Sitecore because:

  • There are existing applications managing the external data, no need to create save function in Sitecore. For example, PIM(Product Information/Inventory Management)
  • External data is huge and ETL might take long time to sync data to Sitecore. For example product inventory data in retail business by SKU, Home Depot sells 300K+, BestBuy sells 725K+; in travel booking system by vacation package, Sunwing sells 10M+
  • External data changes very frequently. For example dynamic price of flight ticket and hotel room, real time merchandise inventory.

So the right approach is,

  • Integrate with external data, no ETL to Sitecore
  • Conduct real time search against external data through Solr

Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene. Installing Solr is easy, see Learn more about Solr.  Here is an example with key points to create a core in Solr and configure index to a table in SQL Server.

solr1

  • solrconfig:
    <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar" /> 
    
    <lib dir="${solr.install.dir:../../../..}/dist/" regex="sqljdbc42\.jar" />
  • managed-schema:
    <field name="ProductID" type="long" indexed="true" required="false" stored="true"/>
    
    <field name="Name" type="string" docValues="true" indexed="true" stored="true"/>
    
    <field name="ProductNumber" type="string" docValues="true" indexed="true" stored="true"/>
  • db-data-config:
    <dataSource driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" url="jdbc:sqlserver://localhost:8983;databaseName=Northwind" user="sa" password="xxxxxx" />
    
    <entity name="item"  query="select * from Products"  deltaQuery="select ProductID from Products and updateDate > '${dataimporter.last_index_time}‘”  deltaImportQuery=“” deletedPkQuery=“” transformer="RegexTransformer,DateFormatTransformer,TemplateTransformer"> </entity>

SolrNet is an Apache Solr client for .NET to interact with the Solr search engine. Sitecore installation package officially supports Lucene for native and Solr for distributed indexing. Since Sitecore 8.2 the SolrNet is included in package which means SolrNet has been fully test by Sitecore. Programming with SolrNet is easy. Here is an example with steps:

  • Initialize:

     

    Startup.Init<Bike>("http://localhost:8983/solr/Bike");    
    
    ISolrOperations<Bike> solr =  ServiceLocator.Current.GetInstance<isolroperations<Bike>>();
  • Query in SolrNet:
    var qSize = new SolrQueryByRange<int>("Size", 40, 52);
    
    var qSubCat = new SolrQueryByField("ProductSubcategory", "Mountain Bikes");
    
    var qColor = new SolrQueryInList("Color", "Silver", "Black");
    
    SolrQueryResults<Bike> bikes = solr.Query(qSize && qSubCat && qColor);
  • Query Converted to Solr REST Api:
    http://localhost:8983/solr/Bike/select?fq=Color:((Silver)%20OR%20(Black))&fq=ProductSubcategory:%20%22Mountain%20Bikes%22&fq=Size:[40%20TO%2052]&indent=on&q=*:*&wt=json
Posted in Information Technology, Sitecore, Solr | Tagged , , , , , | Leave a comment

A trick to set up 301 url redirect in Sitecore

Recently I am working on a task to redirect whole bunch of page urls to a new location. I want to preserve all of the links from search engines. So I need to make sure that the old links give a 301 (Moved Permanently) response back. This will allow the web crawlers to update their databases with the new links as well as keep the old links working in the meantime.

I downloaded the Sitecore 301 redirect module from marketplace. It comes up with three types redirect setting:

  • Exact Match Redirects
  • Regular Expression Redirects
  • Rules based Redirects

It sounds very straightforward. I started to set a redirect with type “Exact Match Redirects”. For example:

http://somesite.com/en/oldpage.aspx –> Sitecore/Content/somesite/newpage.

But no matter how I tried, the redirect just didn’t happen. What’s wrong?

After reading the document and tried several more different scenarios, I found how the redirect module works. Sitecore allows customizing the rendering pipeline which is exactly the redirect module does. This module kicks in right after Sitecore tries to resolve the item for the request. If Sitecore resolves the item, this module doesn’t do anything. Otherwise, it runs through the rules in the above 3 matching order.

One important thing I found is, Sitecore’s language processor resolves the language before rendering happens by retrieving the language flag in url. That means when this redirect modules kicks in to compare the match url, the language has already been solved and the flag in url is removed. For example above old url becomes http://somesite.com/oldpage.aspx. That’s why my previous redirect settings failed. After I removed the language flag from the url, it succeeded.

Another tip to make the redirect work is, make sure the old link is dead/disabled.

Posted in Information Technology, Sitecore | Tagged , , , , | Leave a comment