Current view: XpoLog V7 (Latest). Available: XpoLog V6 and XpoLog V5

Skip to end of metadata
Go to start of metadata

You can create an XML file to build an environment for scanning many servers, and per server, scanning many directories. The path to this XML file is placed in the Add Logs Directories Task, for adding multiple directories to XpoLog, and automating addition of directories.

DirectoryScanner XML General Structure

The following is the XML code of DirectoryScanner.

<DirectoryScanner>
<ScannerNode name="NAME_OF_FOLDER_IN_XPOLOG">
<Account name="ACCOUNT NAME" ... /> <!-- OPTIONAL -->
<ScanDirectories>
<ScanDirectory scanPath="DIRECTORY_PATH_ON_SOURCE">
<ScanConfiguration>
<!-- OPTIONAL CONF PARAMETERS (ScanFileFilter, ScanConfApplications) -->
</ScanConfiguration>
</ScanDirectory>
</ScanDirectories>
</ScannerNode>
</DirectoryScanner>

  The following table describes the general structure of DirectoryScanner. 

Tag PathMandatory/OptionalDescription
DirectoryScannerMandatory 
DirectoryScanner/ScannerNodeMandatoryThe root folder that will be placed above its scanned directories.
DirectoryScanner/AccountOptionalMandatory if XpoLog should connect to a remote server - Windows / UNIX (Creating an Account)
DirectoryScanner/ScanDirectoriesMandatory 
DirectoryScanner/ScanDirectories/ScanDirectoryMandatoryContains the scanPath
DirectoryScanner/ScanDirectories/ScanDirectory/ScanConfigurationOptional 
DirectoryScanner/ScanDirectories/ScanDirectory/ScanConfiguration/ScanFileFilterOptional 
DirectoryScanner/ScanDirectories/ScanDirectory/ScanConfiguration/ScanConfApplicationsOptional 

 XML Reference

ScannerNode Parameter

ParameterMandatory/OptionalDescriptionValues
 name Mandatory

The name of the root folder that will be placed above its scanned directories.
Leave the name empty to create all sub-directories under the parent folder with their original name from the source server.

 String

Example

<ScannerNode name="">

Account Parameters

ParameterMandatory/OptionalDescriptionValues
 nameMandatoryThe account name String
 useEncryptMandatoryIndicates whether or not account password will be encrypted Boolean
 isPublicKeyMandatory (SSH Only)If authentication is done by private key, should be FALSE. 
If user/password are used, it should be TRUE, and a path should be specified under    
privateKeyPath
(see the following parameter).
 Boolean 
 privateKeyPathOptional (SSH Only)The path to the key, if authentication is done by private key 
 isSystemAccount Indicates whether or not account is a system account Boolean
 isScriptAPI Indicates whether or not account is Script API Boolean
 isSSH Indicates whether or not account is SSH Boolean
 isEditable Indicates whether or not account can be edited in the XpoLog Address Book Boolean
 isCertificate Indicates whether or not the account uses a certificate Boolean
 descriptionOptionalDescription of the account 
 classKeyMandatory
Windows: xpolog.eye.media.auth.win.WinAuthenticationAccount
SSH: xpolog.eye.media.telsh.TelnetAccount
 
 certificateIDOptionalThe ID of the certificate, if account uses a certificate (see isCertificate). String
 UserNameMandatory The username that the account uses to connect  String
 isDefault   Boolean
 PortMandatory (SSH Only)The port that will be used to establish the connection to the remote data sourceNumeric
 TYPE_SCP_SFTPOptional (SSH Only)Indicates if the SSH account will use SCP or SFTP (default) protocolString
 PasswordOptionalThe password that the account uses to connect. Optional only if SSH account uses Public/Private keyString
 NetAddressMandatory The IP/hostname of the remote data source used in the accountString

Note 1: If a remote data source is scanned then an account to that source should be specified (it can be verified after execution under XpoLog > Tools > Address Book). In case an account for a specified machine already exists, XpoLog will automatically use it.

Note 2: In case XpoLog is running on a Windows machine, it is recommended to configure a service account on the Windows services panel and then all Windows network logs can be scanned as local without specifying an account in the ScannerNode (path may be \\<server-name>\<drive-name>$\...).

Windows Account Example

<Account name="ACCOUNT NAME" useEncrypt="false" isSystemAccount="false" isEditable="true" description="" classKey="xpolog.eye.media.auth.win.WinAuthenticationAccount" Password="PASSWORD" user="USERNAME" isDefault="false"/>

SSH Account Example

<Account name="ACCOUNT NAME" useEncrypt="false" privateKeyPath="" isSystemAccount="false" isScriptApi="false" isSSH="true" isPublicKey="false" isEditable="true" isCertificate="false" description="" classKey="xpolog.eye.media.telsh.TelnetAccount" certificateId="" UserName="USERNAME" TYPE_SCP_SFTP="SFTP" Port="22" Password="PASSWORD" NetAddress="IP"/>

 

Note: you should not use an id parameter in the account line in the scanner XML. In case an account already exists in XpoLog then based on the NetAdress and Name it will be matched and re-used. In case the account does not exist it will be created during the scanner execution.

ScanDirectory Parameter

ParameterMandatory/OptionalDescriptionValues
 scanPath Mandatory The full path to the directory that is to be scanned Path

Examples

<ScanDirectory scanPath="C:\logs\"> (Windows Local)
<ScanDirectory scanPath="\\qaserver\C$\logs\"> (Windows Network)
<ScanDirectory scanPath="/opt/application/logs/"> (UNIX Local / Over SSH)
 

ScanConfiguration Parameters

ParameterMandatory/OptionalDescriptionValues
 condenseLogsTree  OptionalA "true" value indicates that folders containing only one sub-folder and without logs, will be omitted from the Folders and Logs tree. Boolean
 directoriesToHide OptionalA comma separated list of name expressions of folders that will not be added to the Folders and Logs tree;their sub-folders/logs will be added. 
 fileSuffixesToIgnore  OptionalUnite logs with different suffixes into one log type (advanced)  
 numberOfThreads OptionalThe number of threads to be used as part of the scanning operation Integer
 removeEmptyNodes OptionalIn case there are no matching files under one of the Folders and Logs members, remove it from the Folders and Logs tree. Boolean
 subdirsScanLevel   OptionalThe number of sub-directories to scan from the given directory. Default is unlimited; any number can be specified. Integer or "Unlimited"
 scanMethod  Optional0 = Use existing configuration (file names and content) and automatic matching.
1 = Use existing configuration (file names and content).
2 = Use existing configuration (file names only).
 0, 1, or 2
 namePatternLogic Optional0 = Capture each file separately (without name pattern).
1 = Unite files with a similar names (apply name pattern automatically).
2 = Unite files with a similar suffix (apply name pattern only at the end of the file name).
 0, 1, or 2
 filesToInclude OptionalDefine which files to include in the log scan:
Using wildcard: *.log,*.txt - include all files whose name ends with .log or .txt
Using a regular expression: regexp:\w+\.log - include all files whose name is constructed of word characters only and ends with .log, for example helloWorld.log
 
 filesToExclude OptionalDefine which files to exclude in the log scan:
Using wildcard: *.zip,*.gz - exclude all files whose name ends with .zip or .gz
Using a regular expression: regexp:\w+\.tar\.gz - exclude all files whose name is constructed of word characters only and ends with .tar.gz, for example helloWorld.tar.gz
 
 directoriesToExclude OptionalDefine which directories to exclude from the log scan:
Using wildcard: Apps* - exclude all directories whose name starts with Apps
Using a regular expression: regexp:\d\d\d\d-\d\d\-\d\d - exclude all directories whose name is a date, for example 2013-11-26
 
 directoriesToInclude OptionalDefine which directories to include in the log scan:
Using wildcard: Apps* - include all directories whose name starts with Apps
Using a regular expression: regexp:\d\d\d\d-\d\d\-\d\d - include all directories whose name is a date, for example 2013-11-26
 
templatesToUse OptionalThe scan task will add only logs which were matched to one of the comma separated list of specified templates.String
namePatternToApply OptionalAutomatically name the matched logs based on the given name pattern. Allowed identifiers are:
  • [PARENT_FOLDER n] - the name of n-th parent folder of the log
  • [CHILD_FOLDER n] - the name of n-th child folder of the root folder
  • [APPLICATION] - the name of the log's application
  • [SERVER] - the name of the log's server
  • [LOG] - the current name of the log
String
timeZone OptionalSet the specified time zone on all matched logsString
onlineLogsApplicationOptionalComma separated list of application name(s) that the online logs will be tagged to once createdString
 assignedCollectionPolicyOptionalThe policy name of the collection policy that will be applied on the logs once created;
If the parameter does not exist - the default policy will be automatically applied.
String
fileSuffixesToIgnore  OptionalRegular expression used to ignore part of the files names to define a name pattern String
 enableLogsIndex* Optional True – Online logs that are added under Folders and Logs will be indexed.
False – Disable indexing.
 Boolean
 enableLogsAnalytics* OptionalTrue – Online logs that are added under Folders and Logs will be analyzed by Analytics.
False – Disable Analytics.
Boolean
 addCollectors* OptionalTrue – All the logs that are added by the scanner task will be collected; default policy will be applied, unless a specific policy is specified.
False – Disables logs collection.
 Boolean
enableCollectedLogsAnalytics*  Optional True – enables Analytics analysis on collected logs. Online logs state will be taken from the collection policy;
relevant only when addCollectors="true".
False – Disables Analytics analysis on collected log. 
 Boolean
 enableCollectedLogsIndex*  OptionalTrue – enables indexing of collected logs. The online logs state will be taken from the collection policy;
relevant only when addCollectors="true".
False – Disables indexing of collected logs. 
 Boolean
 collectedLogsApplication*OptionalComma separated list of application name(s) that the collected logs will be tagged to once created;
relevant only when addCollectors="true".
String
* Properties relevant only to versions 4.4 and below

Example

<ScanConfiguration condenseLogsTree="false" directoriesToInclude="log,home" numberOfThreads="3" assignedCollectionPolicy="live" filesToInclude="*.log">

ScanFileFilter Parameters

ParameterMandatory/OptionalDescriptionValues
 timeInterval/timeIntervalUnit  OptionalThe scan will add only log files with last updated time that is within the specified time interval per log type.years, months, weeks, days, hours, mins
 maxNumberOfFiles  OptionalThe maximum number of log files that are added per log type. Integer

Examples

<ScanFileFilter timeInterval="3" timeIntervalUnit="months"/>
<ScanFileFilter maxNumberOfFiles="2"/>

ScanConfApplications Parameters

ParameterMandatory/OptionalDescriptionValues
 applicationNamePattern Optional The pattern that is used to extract the application name. An application will be created as part of the scan process. 
 applicationGroupNamePattern  OptionalThe pattern that is used to extract the application group name. An application will be created as part of the scan process that all its sub-application are tagged to. 

Example

<ScanConfApplications applicationNamePattern="/\w+-(\w+).*" applicationGroupNamePattern="/(\w+)-\w+.*"/>

Templates:

Please use the following examples as templates and modify accordingly (multiple directories per host can be defined by adding more <scanDirecotry> entries / multiple hosts can be defined by adding multiple <scannerNode> entries:

Example 1 (scanner_example_Windows_logs_account_on_xpolog_service): scanner_example_Windows_logs_account_on_xpolog_service.xml

Example 2 (scanner_example_Windows_logs_using_windows_network_account): scanner_example_Windows_logs_using_windows_network_account.xml

Example 3 (scanner_example_Linux_local_logs): scanner_example_Linux_local_logs.xml

Example 4 (scanner_example_Linux_remote_logs): scanner_example_Linux_remote_logs.xml

  • No labels