NaturalDocs::SymbolTable

A package that handles all the gory details of managing symbols.  It handles where they are defined, which files reference them, if any are undefined or duplicated, and loading and saving them to a file.

Usage and Dependencies

Summary
NaturalDocs::SymbolTableA package that handles all the gory details of managing symbols.
Variables
SYMBOLTABLE_FILEHANDLEThe file handle used with SymbolTable.nd.
symbolsA hash of all SymbolStrings.
referencesA hash of all references in the project.
filesA hash of all the files that define symbols and references in the project.
watchedFileA NaturalDocs::SymbolTable::File object of the file being watched for changes.
watchedFileNameThe FileName of the watched file, if any.
watchedFileSymbolDefinitionsA hashref of the symbol definition information for all the SymbolStrings in the watched file.
indexesA hash of generated symbol indexes.
indexChangesA hash of all the indexes that have changed.
indexSectionsWithContentA hash of which sections in an index have content.
rebuildIndexesWhether all indexes should be rebuilt regardless of whether they have been changed.
Files
SymbolTable.ndThe storage file for the symbol table.
IndexInfo.ndThe storage file for information about the indexes.
File Functions
LoadLoads all data files from disk.
LoadSymbolTableLoads SymbolTable.nd from disk.
LoadIndexInfoLoads IndexInfo.nd from disk.
PurgePurges the symbol table of all symbols and references from files that no longer have Natural Docs content.
SaveSaves all data files to disk.
SaveSymbolTableSaves SymbolTable.nd to disk.
SaveIndexInfoSaves IndexInfo.nd to disk.
Modification FunctionsThese functions should not be called after PurgeResolvingInfo().
AddSymbolAdds a symbol definition to the table, if it doesn’t already exist.
AddReferenceAdds a reference to the table, if it doesn’t already exist.
WatchFileForChangesTracks a file to see if any symbols or references were changed or deleted in ways that would require other files to be rebuilt.
AnalyzeChangesHandles any changes found when reparsing a file using WatchFileForChanges().
DeleteReferenceDeletes a reference from the table.
RebuildAllIndexesWhen called, it makes sure all indexes are listed as changed by IndexChanged(), regardless of whether they actually did or not.
PurgeResolvingInfoPurges unnecessary information from the symbol table after it is fully resolved.
PurgeIndexesClears all generated indexes.
Information FunctionsThese functions should not be called until the symbol table is fully resolved.
ReferencesReturns what the passed reference information resolve to, if anything.
LookupReturns information on the passed SymbolString, if it exists.
IndexReturns a symbol index.
HasIndexesDetermines which indexes out of a list actually have content.
IndexChangedReturns whether the specified index has changed.
IndexSectionsWithContentReturns an arrayref of whether each section of the specified index has content.
Event Handlers
OnIndexChangeCalled whenever a change happens to a symbol that would cause an index to be regenerated.
OnInterpretationChangeCalled whenever the current interpretation of a reference changes, meaning it switched from one symbol to another.
OnTargetSymbolChangeCalled whenever the symbol that serves as the interpretation of a reference changes, but the reference still resolves to the same symbol.
Support Functions
DeleteSymbolRemoves a symbol definition from the table.
GenerateInterpretationsGenerates the list of interpretations for the passed reference.
GenerateRelativeInterpretationsGenerates the list of relative interpretations for the passed reference and packages.
SingularInterpretationsOfGenerates singular interpretations of a SymbolString if it can be interpreted as a plural.
AddInterpretationAdds an interpretation to an existing reference.
InterpretReferenceInterprets the passed reference, matching it to the defined symbol with the highest score.
MakeIndexGenerates a symbol index.
SplitSymbolForIndexSplits a SymbolString into its symbol and package portions for indexing.

Variables

SYMBOLTABLE_FILEHANDLE

The file handle used with SymbolTable.nd.

symbols

my %symbols

A hash of all SymbolStrings.  The keys are the SymbolStrings and the values are NaturalDocs::SymbolTable::Symbol objects.

Prior to PurgeResolvingInfo(), both defined symbols and symbols that are merely potential interpretations of references will be here.  Afterwards, only defined symbols will be here.

references

my %references

A hash of all references in the project.  The keys are ReferenceStrings and the values are NaturalDocs::SymbolTable::Reference objects.

Prior to PurgeResolvingInfo(), all possible interpretations will be stored for each reference.  Afterwards, only the current interpretation will be.

files

my %files

A hash of all the files that define symbols and references in the project.  The keys are the FileNames, and the values are NaturalDocs::SymbolTable::File objects.

After PurgeResolvingInfo(), this hash will be empty.

watchedFile

my $watchedFile

A NaturalDocs::SymbolTable::File object of the file being watched for changes.  This is compared to the version in files to see if anything was changed since the last parse.

watchedFileName

my $watchedFileName

The FileName of the watched file, if any.  If there is no watched file, this will be undef.

watchedFileSymbolDefinitions

my %watchedFileSymbolDefinitions

A hashref of the symbol definition information for all the SymbolStrings in the watched file.  The keys are the symbol strings, and the values are NaturalDocs::SymbolTable::SymbolDefinition objects.

indexes

my %indexes

A hash of generated symbol indexes.  The keys are TopicTypes and the values are sorted arrayrefs of NaturalDocs::SymbolTable::IndexElements, or undef if its empty.

indexChanges

my %indexChanges

A hash of all the indexes that have changed.  The keys are the TopicTypes and the entries are undef if they have not changed, or 1 if they have.  The key will not exist if the TopicType has not been checked.

indexSectionsWithContent

my %indexSectionsWithContent

A hash of which sections in an index have content.  The keys are the TopicTypes of each index, and the values are arrayrefs of bools where the first represents symbols, the second numbers, and the rest A-Z.  If there is no information available for an index, it’s entry will not exist here.

rebuildIndexes

my $rebuildIndexes

Whether all indexes should be rebuilt regardless of whether they have been changed.

Files

SymbolTable.nd

The storage file for the symbol table.

Format

[BINARY_FORMAT]
[VersionInt: app version]

The file starts with the standard BINARY_FORMAT VersionInt header.

The first stage of the file is for symbol definitions, analogous to symbols.

[SymbolString: symbol or undef to end] ...

[UInt16: number of definitions]

   [UString16: global definition file] [UString16: TopicType]
      [UString16: prototype] [UString16: summary]

   [UString16: definition file] ...

   ...

These blocks continue until the SymbolString is undef.  Only defined symbols will be included in this file, so number of definitions will never be zero.  The first one is always the global definition.  If a symbol does not have a prototype or summary, the UInt16 length of the string will be zero.

The second stage is for references, which is analogous to references.  Only REFERENCE_TEXT references are stored in this file, and their Resolving Flags are implied so they aren’t stored either.

[ReferenceString (no type, resolving flags): reference or undef to end]

[UInt8: number of definition files]
   [UString16: definition file] [UString16: definition file] ...

These blocks continue until the ReferenceString is undef.  Since there can be multiple using SymbolStrings, those continue until the number of identifiers is zero.  Note that all interpretations are rebuilt rather than stored.

See Also

File Format Conventions

Revisions

1.52

  • Changed AString16s to UString16s.

1.3

  • Symbol TopicTypes were changed from UInt8s to AString16s, now that TopicTypes are strings instead of integer constants.

1.22

  • File format was completely rebuilt to accommodate the new symbol format and to be in binary.  To see the plain text format prior to 1.22, check out 1.21’s version of this file from CVS.  It is too big a change to note here.

IndexInfo.nd

The storage file for information about the indexes.

Format

[Standard Header]

The standard binary file header.

[UString16: index topic name]
[uint8: symbols have content (0 or 1)]
[uint8: numbers have content (0 or 1)]
[uint8: A has content] [uint8: B has content] ...
...

Every index that has information about it is stored with the topic type name first, then 28 uint8s that say whether that part of the index has content or not.  The first is for symbols, the second is for numbers, and the rest are for A-Z.  If an index’s state is unknown, it won’t appear in this file.

Revisions

1.52

  • AString16s were changed to UString16s.

1.4

  • The file is introduced.

File Functions

Load

sub Load

Loads all data files from disk.

LoadSymbolTable

sub LoadSymbolTable

Loads SymbolTable.nd from disk.

LoadIndexInfo

sub LoadIndexInfo

Loads IndexInfo.nd from disk.

Purge

sub Purge

Purges the symbol table of all symbols and references from files that no longer have Natural Docs content.

Save

sub Save

Saves all data files to disk.

SaveSymbolTable

sub SaveSymbolTable

Saves SymbolTable.nd to disk.

SaveIndexInfo

sub SaveIndexInfo

Saves IndexInfo.nd to disk.

Modification Functions

These functions should not be called after PurgeResolvingInfo().

AddSymbol

sub AddSymbol #(symbol,
file,
type,
prototype,
summary)

Adds a symbol definition to the table, if it doesn’t already exist.  If the definition changes or otherwise requires the files that reference it to be updated, the function will call NaturalDocs::Project->RebuildFile() to make sure that they are.

Parameters

symbolThe SymbolString.
fileThe FileName where it’s defined.
typeThe symbol’s TopicType.
prototypeThe symbol’s prototype, if applicable.
summaryThe symbol’s summary, if applicable.

AddReference

sub AddReference #(type,
symbol,
scope,
using,
file,
resolvingFlags) or (referenceString, file)

Adds a reference to the table, if it doesn’t already exist.

Parameters

typeThe ReferenceType.
symbolThe reference SymbolString.
scopeThe scope SymbolString it appears in.
usingAn arrayref of scope SymbolStrings accessible to the reference via “using” statements, or undef if none.
fileThe FileName where the reference appears.  This is not required unless the type is REFERENCE_TEXT.
resolvingFlagsThe Resolving Flags of the reference.  They will be ignored if the type is REFERENCE_TEXT.

Alternate Parameters

referenceStringThe ReferenceString to add.
fileThe FileName where the reference appears.  This is not required unless the type is REFERENCE_TEXT.

WatchFileForChanges

sub WatchFileForChanges #(file)

Tracks a file to see if any symbols or references were changed or deleted in ways that would require other files to be rebuilt.  Assumes that after this function call, the entire file will be parsed again, and thus every symbol and reference will go through AddSymbol() and AddReference().  Afterwards, call AnalyzeChanges() to handle any differences.

Parameters

fileThe FileName to watch.

AnalyzeChanges

sub AnalyzeChanges

Handles any changes found when reparsing a file using WatchFileForChanges().

DeleteReference

sub DeleteReference #(referenceString,
file)

Deletes a reference from the table.

Be careful with this function, as deleting a reference means there are no more of them in the file at all.  The tables do not keep track of how many times references appear in a file.  In these cases you should instead call WatchFileForChanges(), reparse the file, thus readding all the references, and call AnalyzeChanges().

REFERENCE_TEXT references should always be managed with WatchFileForChanges() and AnalyzeChanges().  This function should only be used externally for other types of references.

Parameters

referenceStringThe ReferenceString.
fileThe FileName where the reference is.  This is not required unless the type is REFERENCE_TEXT.

RebuildAllIndexes

sub RebuildAllIndexes

When called, it makes sure all indexes are listed as changed by IndexChanged(), regardless of whether they actually did or not.

This can be called at any time.

PurgeResolvingInfo

sub PurgeResolvingInfo

Purges unnecessary information from the symbol table after it is fully resolved.  This will reduce the memory footprint for the build stage.  After calling this function, you can only call the Information Functions and Save().

PurgeIndexes

sub PurgeIndexes

Clears all generated indexes.

Information Functions

These functions should not be called until the symbol table is fully resolved.

References

sub References #(type,
symbol,
scope,
using,
file,
resolvingFlags) or (referenceString, file)

Returns what the passed reference information resolve to, if anything.  Note that this only works if the reference had been previously added to the table via AddReference() with the exact same parameters.

Parameters

typeThe ReferenceType.
symbolThe reference SymbolString.
scopeThe scope SymbolString the reference appears in, or undef if none.
usingAn arrayref of scope SymbolStrings available to the reference via using statements.
fileThe source FileName the reference appears in, or undef if none.
resolvingFlagsThe Resolving Flags of the reference.  Ignored if the type is REFERENCE_TEXT.

Alternate Parameters

referenceStringThe ReferenceString to resolve.
fileThe source FileName the reference appears in, or undef if none.

Returns

A NaturalDocs::SymbolTable::ReferenceTarget object, or undef if the reference doesn’t resolve to anything.

Lookup

sub Lookup #(symbol,
file)

Returns information on the passed SymbolString, if it exists.  Note that the symbol must be fully resolved.

Parameters

symbolThe SymbolString.
fileThe source FileName the reference appears in, or undef if none.

Returns

A NaturalDocs::SymbolTable::ReferenceTarget object, or undef if the symbol isn’t defined.

Index

sub Index #(type)

Returns a symbol index.

Indexes are generated on demand, but they are stored so subsequent calls for the same index will be fast.  Call PurgeIndexes() to clear the generated indexes.

Parameters

typeThe TopicType of symbol to limit the index to, or undef for none.

Returns

An arrayref of sections.  The first represents all the symbols, the second the numbers, and the rest A through Z.  Each section is a sorted arrayref of NaturalDocs::SymbolTable::IndexElement objects.  If a section has no content, it will be undef.

HasIndexes

sub HasIndexes #(types)

Determines which indexes out of a list actually have content.

Parameters

typesAn existence hashref of the TopicTypes to check for indexes.

Returns

An existence hashref of all the specified indexes that have content.  Will return an empty hashref if none.

IndexChanged

sub IndexChanged #(TopicType type)

Returns whether the specified index has changed.

Parameters

typeThe TopicType to limit the index to.

IndexSectionsWithContent

sub IndexSectionsWithContent #(TopicType type)

Returns an arrayref of whether each section of the specified index has content.  The first entry will be for symbols, the second for numbers, and the rest A-Z.  Do not change the arrayref.

Event Handlers

OnIndexChange

sub OnIndexChange #(TopicType type)

Called whenever a change happens to a symbol that would cause an index to be regenerated.

Parameters

typeThe TopicType of the symbol that caused the change.

OnInterpretationChange

sub OnInterpretationChange #(referenceString)

Called whenever the current interpretation of a reference changes, meaning it switched from one symbol to another.

Parameters

referenceStringThe ReferenceString whose current interpretation changed.

OnTargetSymbolChange

sub OnTargetSymbolChange #(referenceString)

Called whenever the symbol that serves as the interpretation of a reference changes, but the reference still resolves to the same symbol.  This would happen if the type, prototype, summary, or which file serves as global definition of the symbol changes.

Parameters

referenceStringThe ReferenceString whose interpretation’s symbol changed.

Support Functions

DeleteSymbol

sub DeleteSymbol #(symbol,
file)

Removes a symbol definition from the table.  It will call OnInterpretationChange() for all references that have it as their current interpretation.

External code should not attempt to delete symbols using this function.  Instead it should call <WatchFileFoChanges()>, reparse the file, and call AnalyzeChanges().

Parameters

symbolThe SymbolString.
fileThe FileName where the definition is.

GenerateInterpretations

sub GenerateInterpretations #(referenceString)

Generates the list of interpretations for the passed reference.  Also creates potential symbols as necessary.

Parameters

referenceStringThe ReferenceString to generate the interpretations of.

GenerateRelativeInterpretations

sub GenerateRelativeInterpretations #(referenceString,
symbol,
singulars,
package,
score,
dontUseFull)

Generates the list of relative interpretations for the passed reference and packages.  Also creates potential symbols as necessary.

This function will not create global interpretations.  It will create a local interpretations (symbol + all packages) unless you set dontUseFull.

Parameters

referenceStringThe ReferenceString to generate interpretations for.
symbolThe SymbolString to generate interpretations of.
singularsA reference to an array of singular SymbolStrings to also generate interpretations of.  Set to an empty array if none.
packageThe package SymbolString to use.  May be undef.
scoreThe starting score to apply.
dontUseFullWhether to not generate an interpretation including the full package identifier.  If set, generated interpretations will start one level down.

Returns

The next unused score.  This is basically the passed score minus the number of interpretations created.

SingularInterpretationsOf

sub SingularInterpretationsOf #(symbol)

Generates singular interpretations of a SymbolString if it can be interpreted as a plural.  Not all of them will be valid singular forms, but that doesn’t matter since it’s incredibly unlikely an invalid form would exist as a symbol.  What matters is that the legimate singular is present on the list.

Parameters

symbolThe SymbolString.

Returns

An array of potential singular interpretations as SymbolStrings, in no particular order.  If the symbol can’t be interpreted as a plural, returns an empty array.

AddInterpretation

sub AddInterpretation #(referenceString,
symbol,
score)

Adds an interpretation to an existing reference.  Creates potential symbols as necessary.

Parameters

referenceStringThe ReferenceString to add the interpretation to.
symbolThe SymbolString the reference can be interpreted as.
scoreThe score of the interpretation.

InterpretReference

sub InterpretReference #(referenceString)

Interprets the passed reference, matching it to the defined symbol with the highest score.  If the symbol is already interpreted, it will reinterpret it.  If there are no matches, it will make it an undefined reference.

Parameters

referenceStringThe ReferenceString to interpret.

MakeIndex

sub MakeIndex #(type)

Generates a symbol index.

Parameters

typeThe TopicType to limit the index to.

Returns

An arrayref of sections.  The first represents all the symbols, the second the numbers, and the rest A through Z.  Each section is a sorted arrayref of NaturalDocs::SymbolTable::IndexElement objects.  If a section has no content, it will be undef.

SplitSymbolForIndex

sub SplitSymbolForIndex #(symbol,
type)

Splits a SymbolString into its symbol and package portions for indexing.

Parameters

symbolThe SymbolString.
typeIts TopicType.

Returns

The array ( symbol, package ), which are both SymbolStrings.  If the symbol is global, package will be undef.

The storage file for the symbol table.
my %symbols
A hash of all SymbolStrings.
A scalar which encodes a normalized array of identifier strings representing a full or partially-resolved symbol.
my %references
A hash of all references in the project.
my %files
A hash of all the files that define symbols and references in the project.
my $watchedFile
A NaturalDocs::SymbolTable::File object of the file being watched for changes.
A class representing a file, keeping track of what symbols and references are defined in it.
my $watchedFileName
The FileName of the watched file, if any.
A string representing the absolute, platform-dependent path to a file.
my %watchedFileSymbolDefinitions
A hashref of the symbol definition information for all the SymbolStrings in the watched file.
my %indexes
A hash of generated symbol indexes.
my %indexChanges
A hash of all the indexes that have changed.
my %indexSectionsWithContent
A hash of which sections in an index have content.
my $rebuildIndexes
Whether all indexes should be rebuilt regardless of whether they have been changed.
sub Load
Loads all data files from disk.
sub LoadSymbolTable
Loads SymbolTable.nd from disk.
sub LoadIndexInfo
Loads IndexInfo.nd from disk.
The storage file for information about the indexes.
sub Purge
Purges the symbol table of all symbols and references from files that no longer have Natural Docs content.
sub Save
Saves all data files to disk.
sub SaveSymbolTable
Saves SymbolTable.nd to disk.
sub SaveIndexInfo
Saves IndexInfo.nd to disk.
sub PurgeResolvingInfo
Purges unnecessary information from the symbol table after it is fully resolved.
sub AddSymbol #(symbol,
file,
type,
prototype,
summary)
Adds a symbol definition to the table, if it doesn’t already exist.
sub AddReference #(type,
symbol,
scope,
using,
file,
resolvingFlags) or (referenceString, file)
Adds a reference to the table, if it doesn’t already exist.
sub WatchFileForChanges #(file)
Tracks a file to see if any symbols or references were changed or deleted in ways that would require other files to be rebuilt.
sub AnalyzeChanges
Handles any changes found when reparsing a file using WatchFileForChanges().
sub DeleteReference #(referenceString,
file)
Deletes a reference from the table.
sub RebuildAllIndexes
When called, it makes sure all indexes are listed as changed by IndexChanged(), regardless of whether they actually did or not.
sub IndexChanged #(TopicType type)
Returns whether the specified index has changed.
sub PurgeIndexes
Clears all generated indexes.
sub References #(type,
symbol,
scope,
using,
file,
resolvingFlags) or (referenceString, file)
Returns what the passed reference information resolve to, if anything.
sub Lookup #(symbol,
file)
Returns information on the passed SymbolString, if it exists.
sub Index #(type)
Returns a symbol index.
sub HasIndexes #(types)
Determines which indexes out of a list actually have content.
sub IndexSectionsWithContent #(TopicType type)
Returns an arrayref of whether each section of the specified index has content.
sub OnIndexChange #(TopicType type)
Called whenever a change happens to a symbol that would cause an index to be regenerated.
sub OnInterpretationChange #(referenceString)
Called whenever the current interpretation of a reference changes, meaning it switched from one symbol to another.
sub OnTargetSymbolChange #(referenceString)
Called whenever the symbol that serves as the interpretation of a reference changes, but the reference still resolves to the same symbol.
sub DeleteSymbol #(symbol,
file)
Removes a symbol definition from the table.
sub GenerateInterpretations #(referenceString)
Generates the list of interpretations for the passed reference.
sub GenerateRelativeInterpretations #(referenceString,
symbol,
singulars,
package,
score,
dontUseFull)
Generates the list of relative interpretations for the passed reference and packages.
sub SingularInterpretationsOf #(symbol)
Generates singular interpretations of a SymbolString if it can be interpreted as a plural.
sub AddInterpretation #(referenceString,
symbol,
score)
Adds an interpretation to an existing reference.
sub InterpretReference #(referenceString)
Interprets the passed reference, matching it to the defined symbol with the highest score.
sub MakeIndex #(type)
Generates a symbol index.
sub SplitSymbolForIndex #(symbol,
type)
Splits a SymbolString into its symbol and package portions for indexing.
A package to handle the command line and various other program settings.
A package to manage all the programming languages Natural Docs supports.
A package that manages information about the files in the source tree, as well as the list of files that have to be parsed and built.
These functions should not be called until the symbol table is fully resolved.
The reference appears in the text of the documentation.
sub ParseForInformation #(file)
Parses the input file for information.
These functions should not be called after PurgeResolvingInfo().
A class representing a symbol or a potential symbol.
All the information about a reference that makes it unique encoded into a string.
A class representing a symbol or a potential symbol.
A class representing a symbol definition.
A string representing a topic type as defined in Topics.txt.
A class representing part of an indexed symbol.
An 8-bit constant that’s used as the first byte of binary data files.
A comparable integer representing a version number.
Used to influence the method of resolving references in NaturalDocs::SymbolTable.
sub RebuildFile #(file)
Adds the file to the list of files to build.
The type of a reference.
A class for storing information about a reference target.
Close