0-byte and 1-byte files in workflow directory

As content is created and moves through workflow, editors can make notes in the "Action Taken" and "Action To Take" fields. These notes are stored in urlcolumns in Assignment, and so appear as files in workflow directory, which is the "defdir" for the Assignment table.

In some earlier versions of CS there is an issue whereby if an editor does not provide a comment with their action, a 0-byte or 1-byte file is still created anyway. If workflow is heavily used, then many of these files may be created, causing:

• consumption of inodes on the shared disk
• performance impact, as some workflow tags used by the UI will read these files whilst checking assignments

In one production system, in the space of one year since go-live 260'000 0-byte and 47'000 1-byte files were created, and editorial performance had noticeably dropped as a result.

This issue is resolved from CS 7.5.0 patch 5 onwards, 0-byte and 1-byte files are no longer created although though the fix will not remove any existing ones.

It is possible to remove these 0-byte and 1-byte files, but since references exist to them from the Assignment table, you must remove those references too. Here is a script that can clean them up. Take a full backup first, then change ‘workflowdir’ in the script to point to the correct workflow folder.  The script will output two things:

• A SQL script "aupdate.sql" to run on the db, that will remove the references to the 0-byte and 1-byte files
• Another shell script "adelete.sh" that will actually delete 0-byte and 1-byte files

After running aupdate.sql and adelete.sh, you will need to restart (or flush resultset cache on Assignment table).

#!/bin/sh

workflowdir=/home/csuser/Shared/workflow
rm -f /tmp/aupdate.sql
rm -f /tmp/adelete.sh

cd $workflowdir
find . -size 0c > /tmp/0bytelist
find . -size 1c > /tmp/1bytelist
for i in `cat /tmp/0bytelist /tmp/1bytelist`; do
  filename=`echo $i | cut -c3-`
  echo "update Assignment set urlassigncomment = '' where urlassigncomment = '$filename'  " >> /tmp/aupdate.sql
  echo "update Assignment set urlgroupcomment = '' where urlgroupcomment = '$filename'  " >> /tmp/aupdate.sql
  echo "update Assignment set urlclearcomment = '' where urlclearcomment = '$filename'  " >> /tmp/aupdate.sql
  echo "rm -f $workflowdir$filename" >> /tmp/adelete.sh
done

echo Found `wc -l /tmp/0bytelist` 0-byte files in workflow
echo Found `wc -l /tmp/1bytelist` 1-byte files in workflow
echo Run /tmp/aupdate.sql in sqlplus
echo Run /tmp/adelete.sh in the shell 

Content-Disposition filename on chrome

If you are seeing this in Chrome when fetching a blob from BlobServer or SatelliteServer

The response from the server contained duplicate headers. This problem is generally the result of a misconfigured website or proxy. Only the website or proxy administrator can fix this issue.
Error 349 (net::ERR_RESPONSE_HEADERS_MULTIPLE_CONTENT_DISPOSITION): Multiple Content-Disposition headers received. This is disallowed to protect against HTTP response splitting attacks.

using a url like

http://host:port/cs/Satellite?blobcol=urldata&blobkey=id&blobtable=MungoBlobs&blobwhere=...

then the issue may be that there is a Content-Disposition header being sent back from BlobServer/Satellite, which is setting a filename for the download, and the filename has a comma in it. e.g.

Content-Disposition: attachment; filename=image,5.jpg
Content-Type: image/jpeg

Chrome doesn't like the presence of the comma, although it is not strictly speaking a bug (See http://code.google.com/p/chromium/issues/detail?id=103618). Chrome is interpreting the HTTP RFC to the letter by expecting the filename to be double quote escaped due to the presence of the comma, like so:

Content-Disposition: attachment; filename="image,5.jpg"
Content-Type: image/jpeg

The tags satellite:blob, render:satelliteblob or render:getbloburl are where Content-Disposition header is set, in the template code, so adjust this to ensure that the filename value is double quoted, or encode it another way. This solution by S. James is to use URLEncoder:

<satellite:argument name="blobheadervalue1" value='<%="inline; filename=" + URLEncoder.encode( ics.GetVar("BlobFileName"),"UTF-8") %>'/>

Unfortunately browsers vary in the way they interpret Content-Disposition header, WebCenter Sites does not get involved with attempting to negotiate the best value for a browser, it will just set what the developer chooses to set, so it is a developer challenge to find the best Content-Disposition values to use. A good reference for what works on different browsers is http://greenbytes.de/tech/tc2231/.

Sites 11gR1

Oracle WebCenter Sites 11.1.1.6.0 aka 11gR1 (formerly FatWire ContentServer) is now available. This release provides the completely new modern web authoring interface that replaces the Dash UI, and a new facelift to the Advanced UI.

More info is being posted to the WebCenter blog, and all documentation can now be found on Oracle Technology Network (OTN).

Excellent tool for visualizing garbage collection

IBM's PMAT (Pattern Modelling and Analysis Tool) is excellent for visualizing JVM garbage collection: https://www.ibm.com/developerworks/mydeveloperworks/groups/service/html/communityview?communityUuid=22d56091-3a7b-4497-b36e-634b51838e11.

It can parse logs with verbose:gc or Xverbosegc enabled on IBM JVM 1.3.x,1.4.x, 5.0, and 6.0; Sun JVM 1.4.1/1.4.2/5.0/6.0; or HP-UX JVM 1.4.1/1.4.2/5.0/6.0.

Sysinternals Tools

For troubleshooting issues on Windows, the very best free tools can be found in one place - Sysinternals (now owned by Microsoft, and hosted in Technet). These tools, written by Mark Russinovich and Bryce Cogswell are elegant, efficient, and always useful when diagnosing or troubleshooting issues on Windows.

To give one simple example; Why is my PC so slow? Well, procexp shows low CPU but total disk I/O delta shows a few processes being very active, and procmon lets you see detailed file/network/registry access for all or specific processes. What it told me:

• Snarl system monitor extension was too intensive checking the network status.
• GoToMeetinglauncher was obsessed with constantly reading file attributes for all running processes, even though no meeting was underway.
• TrueImageMonitor was perpetually writing it's SystemState.xml.
• ExistDB's wrapper.log had been left in debug mode and was writing too much
• htcUPCTloader was perpetually recreating it's detecteddevice.xml and detecteddevice_resp.xml files.
• Oracle DB was continually writing an error into alertXE.log
• Windows SearchProtocolHost was trying to digest a 2gb db file which is written by the Oracle Beehive Outlook extension, despite being told to ignore everything under c:\Users\...\AppData
• McShield and Windows Search were perpetually re-reading various files of the above.

Log tailing on Windows

Unsolicited testimonial:

Best log tailing application on Windows that I have tried, and I've tried them all - http://www.hootech.com/WinTail/

Log hygiene

Some general recommendations regarding logging to futuretense.txt

• If the log is filling up with noise, resist the urge to suppress loggers to ERROR, WARN or FATAL without knowing what they are telling you. If something is frequently seen in the log then you should try to understand what it means, to know if action is needed. Only suppress loggers when you know the messages are harmless. At INFO level, CS is typically not very noisy.
  
  
• If you don't know what an error message is telling you, try to find out by checking the documented errorcodes or examining the stack trace. If you can't find out what a message means, ask Support.

• As of CS7.6, log4j is an option during installation, and is in most cases preferable over commons-logging. Log4j is more configurable, and you can change log levels on the fly without requiring a restart. One downside to log4j is that there is no out-of-the-box way to have per-ip logfiles, it requires some extra work.

• Log4j lets you define the format of log messages, such as:
  
  log4j.appender.FWDefaultAppender.layout.ConversionPattern=[%d][%t][%c][%p] %m%n
  
  This example format includes the most useful information, in particular the full name of the logger, and the thread ID that originated the message. When many thread are writing to the same log, you can separate out message using the thread ID.
  

• Consider using a new logger for any customization rather than relying on the default logger which is "com.fatwire.logging.cs". The ics:logmsg tag can let you specify the name of a logger, which you can define in log4j.properties or commons-logging.properties and so doing isolate your customization's logging from anything CS writes.
  
  
• Set the log rolling to something reasonable, e.g. sufficient to capture the whole day's activity and archive it. You may want to refer back to it for troubleshooting. You can configure the log to roll on a particular size threshold, and how many copies to keep.