Sysinternals Tools

For troubleshooting issues on Windows, the very best free tools can be found in one place - Sysinternals (now owned by Microsoft, and hosted in Technet). These tools, written by Mark Russinovich and Bryce Cogswell are elegant, efficient, and always useful when diagnosing or troubleshooting issues on Windows.

To give one simple example; Why is my PC so slow? Well, procexp shows low CPU but total disk I/O delta shows a few processes being very active, and procmon lets you see detailed file/network/registry access for all or specific processes. What it told me:

• Snarl system monitor extension was too intensive checking the network status.
• GoToMeetinglauncher was obsessed with constantly reading file attributes for all running processes, even though no meeting was underway.
• TrueImageMonitor was perpetually writing it's SystemState.xml.
• ExistDB's wrapper.log had been left in debug mode and was writing too much
• htcUPCTloader was perpetually recreating it's detecteddevice.xml and detecteddevice_resp.xml files.
• Oracle DB was continually writing an error into alertXE.log
• Windows SearchProtocolHost was trying to digest a 2gb db file which is written by the Oracle Beehive Outlook extension, despite being told to ignore everything under c:\Users\...\AppData
• McShield and Windows Search were perpetually re-reading various files of the above.

Log tailing on Windows

Unsolicited testimonial:

Best log tailing application on Windows that I have tried, and I've tried them all - http://www.hootech.com/WinTail/

Log hygiene

Some general recommendations regarding logging to futuretense.txt

• If the log is filling up with noise, resist the urge to suppress loggers to ERROR, WARN or FATAL without knowing what they are telling you. If something is frequently seen in the log then you should try to understand what it means, to know if action is needed. Only suppress loggers when you know the messages are harmless. At INFO level, CS is typically not very noisy.
  
  
• If you don't know what an error message is telling you, try to find out by checking the documented errorcodes or examining the stack trace. If you can't find out what a message means, ask Support.

• As of CS7.6, log4j is an option during installation, and is in most cases preferable over commons-logging. Log4j is more configurable, and you can change log levels on the fly without requiring a restart. One downside to log4j is that there is no out-of-the-box way to have per-ip logfiles, it requires some extra work.

• Log4j lets you define the format of log messages, such as:
  
  log4j.appender.FWDefaultAppender.layout.ConversionPattern=[%d][%t][%c][%p] %m%n
  
  This example format includes the most useful information, in particular the full name of the logger, and the thread ID that originated the message. When many thread are writing to the same log, you can separate out message using the thread ID.
  

• Consider using a new logger for any customization rather than relying on the default logger which is "com.fatwire.logging.cs". The ics:logmsg tag can let you specify the name of a logger, which you can define in log4j.properties or commons-logging.properties and so doing isolate your customization's logging from anything CS writes.
  
  
• Set the log rolling to something reasonable, e.g. sufficient to capture the whole day's activity and archive it. You may want to refer back to it for troubleshooting. You can configure the log to roll on a particular size threshold, and how many copies to keep.

Empty directories in SystemPageCache

Empty directories can grow under the SystemPageCache on the shared disk. This can contribute to unnecessary consumption of inodes in production where there is a high turnover of cached items. Cached pagelets are stored in subfolders under SystemPageCache, and when CS removes a pagelet the *.hdr, *.txt and *.qry files are deleted but the empty directories are not. These can be purged, preferably during a downtime.

E.g.;

find /path/to/SystemPageCache -type d -empty -depth -exec rmdir {} \;

Yum repositories on OEL6

OEL6 doesn't come with any yum repositories enabled, but there is a public yum server that can be used. Instructions for enabling it in /etc/yum.repos.d are here http://public-yum.oracle.com/

Installing on Windows 7 and other non-supported stacks

When installing on Windows 7, you are likely to get a NullPointerException before the splash screen. This can be avoided by setting:

-Dos.name="Windows Server 2008"

to the csInstall.bat or csInstall.sh.

Also if you spend any time installing or working on unsupported stacks, you may like to know that some unsupported stacks are pre-configured in the cs-core.xml but commented out. For example, HSQLDB as a database choice exists, but it is commented out. Similarly a "generic" J2EE application server choice exists, but it commented out. Installing on unsupported stacks is of course strongly discouraged for anything except a test, and no official help exists for such stacks.

Also good to know, is that all the tags found in the installer control files like cscore.xml and csinstallbase.xml are documented in the csinstall.dtd, with text describing their function and inputs.

Tweaking resultset cache on revision tracking tables

As of CS7.6 It's not documented yet, but in futuretense.ini you can tweak the resultset cache settings for the hidden revision tracking tables e.g; "Content_C_t". These hidden tables are not visible in CSE, or the Support Tools resultset cache profiler.

The syntax is as you would expect:

cc.Content_C_tCSz=1000
cc.Content_C_tTimeout=15

Disclaimer

I am an employee of Oracle. I am not an official spokesperson for Oracle. The views expressed on this site are my own and do not necessarily reflect the views of Oracle.