NaturalDocs::SourceDB

SourceDB is an experimental package meant to unify the tracking of various elements in the source code.

Requirements

Summary
NaturalDocs::SourceDBSourceDB is an experimental package meant to unify the tracking of various elements in the source code.
The IdeaFor quite a while Natural Docs only needed SymbolTable.
AssumptionsSourceDB is built around certain assumptions.
Types
ExtensionIDA unique identifier for each NaturalDocs::SourceDB extension as given out by RegisterExtension().
Variables
extensionsAn array of NaturalDocs::SourceDB::Extension-derived extensions, as added with RegisterExtension().
extensionUsesDefinitionObjectsAn array where the indexes are ExtensionIDs and the values are whether that extension uses its own definition class derived from NaturalDocs::SourceDB::ItemDefinition or it just tracks their existence.
itemsThe array of source items.
filesA hashref mapping source FileNames to NaturalDocs::SourceDB::Files.
watchedFileWhen a file is being watched for changes, will be a NaturalDocs::SourceDB::File for that file.
watchedFileNameWhen a file is being watched for changes, will be the FileName of the file being watched.
watchedFileDefinitionsWhen a file is being watched for changes, will be a NaturalDocs::SourceDB::WatchedFileDefinitions object.
Extension Functions
RegisterExtensionRegisters a NaturalDocs::SourceDB::Extension-derived package and returns a unique ExtensionID for it.
File Functions
LoadLoads the data of the source database and all the extensions.
SaveSaves the data of the source database and all its extensions.
PurgeDeletedSourceFilesRemoves all data associated with deleted source files.
Item Functions
AddItemAdds the passed item to the database.
GetItemReturns the NaturalDocs::SourceDB::Item-derived object for the passed ExtensionID and item string, or undef if there is none.
DeleteItemDeletes the record of the passed ExtensionID and item string.
HasItemReturns whether there is an item defined for the passed ExtensionID and item string.
GetAllItemsHashRefReturns a hashref of all the items defined for an extension.
Definition Functions
AddDefinitionAdds a definition to an item.
ChangeDefinitionChanges the definition of an item.
GetDefinitionIf the extension uses custom NaturalDocs::SourceDB::ItemDefinition classes, returns it for the passed definition or undef if it doesn’t exist.
DeleteDefinitionRemoves the definition for the passed item.
HasDefinitionsReturns whether there are any definitions for this item.
HasDefinitionReturns whether there is a definition for the passed FileName.
Watched File Functions
WatchFileForChangesBegins watching a file for changes.
WatchingFileForChangesReturns whether we’re currently watching a file for changes or not.
AnalyzeWatchedFileChangesAnalyzes the watched file for changes.

The Idea

For quite a while Natural Docs only needed SymbolTable.  However, 1.3 introduced the ClassHierarchy package which duplicated some of its functionality to track classes and parent references.  1.4 now needs ImageReferenceTable, so this package was an attempt to isolate the common functionality so the wheel doesn’t have to keep being rewritten as the scope of Natural Docs expands.

SourceDB is designed around Extensions and items.  The purposefully vague “items” are anything in the source code that we need to track the definitions of.  Extensions are the packages to track them, only they’re derived from NaturalDocs::SourceDB::Extension and registered with this package instead of being free standing and duplicating functionality such as watched files.

The architecture on this package isn’t comprehensive yet.  As more extensions are added or previously made free standing packages are migrated to it it will expand to encompass them.  However, it’s still experimental so this concept may eventually be abandoned for something better instead.

Assumptions

SourceDB is built around certain assumptions.

One item per file

SourceDB assumes that only the first item per file with a particular item string is relevant.  For example, if two functions have the exact same name, there’s no way to link to the second one either in HTML or internally so it doesn’t matter for our purposes.  Likewise, if two references are exactly the same they go to the same target, so it doesn’t matter whether there’s one or two or a thousand.  All that matters is that at least one reference exists in this file because you only need to determine whether the entire file gets rebuilt.  If two items are different in some meaningful way, they should generate different item strings.

Watched file parsing

SourceDB assumes the parse method is that the information that was stored from Natural Docs’ previous run is loaded, a file is watched, that file is reparsed, and then AnalyzeWatchedFileChanges() is called.  When the file is reparsed all items within it are added the same as if the file was never parsed before.

If there’s a new item this time around, that’s fine no matter what.  However, a changed item wouldn’t normally be recorded because the previous run’s definition is seen as the first one and subsequent ones are ignored.  Also, deleted items would normally not be recorded either because we’re only adding.

The watched file method fixes this because everything is also added to a second, clean database specifically for the watched file.  Because it starts clean, it always gets the first definition from the current parse which can then be compared to the original by AnalyzeWatchedFileChanges().  Because it starts clean you can also compare it to the main database to see if anything was deleted, because it would appear in the main database but not the watched one.

This means that functions like ChangeDefinition() and DeleteDefinition() should only be called by AnalyzeWatchedFileChanges().  Externally only AddDefinition() should be called.  DeleteItem() is okay to be called externally because entire items aren’t managed by the watched file database, only definitions.

Types

ExtensionID

A unique identifier for each NaturalDocs::SourceDB extension as given out by RegisterExtension().

Variables

extensions

my @extensions

An array of NaturalDocs::SourceDB::Extension-derived extensions, as added with RegisterExtension().  The indexes are the ExtensionIDs and the values are package references.

extensionUsesDefinitionObjects

my @extensionUsesDefinitionObjects

An array where the indexes are ExtensionIDs and the values are whether that extension uses its own definition class derived from NaturalDocs::SourceDB::ItemDefinition or it just tracks their existence.

items

my @items

The array of source items.  The ExtensionIDs are the indexes, and the values are hashrefs mapping the item string to NaturalDocs::SourceDB::Item-derived objects.  Hashrefs may be undef.

files

my %files

A hashref mapping source FileNames to NaturalDocs::SourceDB::Files.

watchedFile

my $watchedFile

When a file is being watched for changes, will be a NaturalDocs::SourceDB::File for that file.  Is undef otherwise.

When the file is parsed, items are added to both this and the version in files.  Thus afterwards we can compare the two to see if any were deleted since the last time Natural Docs was run, because they would be in the files version but not this one.

watchedFileName

my $watchedFileName

When a file is being watched for changes, will be the FileName of the file being watched.  Is undef otherwise.

watchedFileDefinitions

my $watchedFileDefinitions

When a file is being watched for changes, will be a NaturalDocs::SourceDB::WatchedFileDefinitions object.  Is undef otherwise.

When the file is parsed, items are added to both this and the version in items.  Since only the first definition is kept, this will always have the definition info from the file whereas the version in items will have the first definition as of the last time Natural Docs was run.  Thus they can be compared to see if the definitions of items that existed the last time around have changed.

Extension Functions

RegisterExtension

sub RegisterExtension #(package extension,
bool usesDefinitionObjects) => ExtensionID

Registers a NaturalDocs::SourceDB::Extension-derived package and returns a unique ExtensionID for it.  All extensions must call this before they can be used.

Registration Order

The order in which extensions register is important.  Whenever possible, items are added in the order their extensions registered.  However, items are changed and deleted in the reverse order.  Take advantage of this to minimize churn between extensions that are dependent on each other.

For example, when symbols are added or deleted they may cause references to be retargeted and thus their files need to be rebuilt.  However, adding or deleting references never causes the symbols’ files to be rebuilt.  So it makes sense that symbols should be created before references, and that references should be deleted before symbols.

Parameters

extensionThe package or object of the extension.  Must be derived from NaturalDocs::SourceDB::Extension.
usesDefinitionObjectsWhether the extension uses its own class derived from NaturalDocs::SourceDB::ItemDefinition or simply tracks each definitions existence.

Returns

An ExtensionID unique to the extension.  This should be saved because it’s required in functions such as AddItem().

File Functions

Load

sub Load

Loads the data of the source database and all the extensions.  Will call NaturalDocs::SourceDB::Extension->Load() for all of them, unless there’s a situation where all the source files are going to be reparsed anyway in which case it’s not needed.

Save

sub Save

Saves the data of the source database and all its extensions.  Will call NaturalDocs::SourceDB::Extension->Save() for all of them.

PurgeDeletedSourceFiles

sub PurgeDeletedSourceFiles

Removes all data associated with deleted source files.

Item Functions

AddItem

sub AddItem #(ExtensionID extension,
string itemString,
NaturalDocs::SourceDB::Item item) => bool

Adds the passed item to the database.  This will not work if the item string already exists.  The item added should not already have definitions attached.  Only use this to add blank items and then call AddDefinition() instead.

Parameters

extensionAn ExtensionID.
itemStringThe string serving as the item identifier.
itemAn object derived from NaturalDocs::SourceDB::Item.

Returns

Whether the item was added, that is, whether it was the first time this item was added.

GetItem

sub GetItem #(ExtensionID extension,
string itemString) => bool

Returns the NaturalDocs::SourceDB::Item-derived object for the passed ExtensionID and item string, or undef if there is none.

DeleteItem

sub DeleteItem #(ExtensionID extension,
string itemString) => bool

Deletes the record of the passed ExtensionID and item string.  Do not delete items that still have definitions.  Use DeleteDefinition() first.

Parameters

extensionThe ExtensionID.
itemStringThe item’s identifying string.

Returns

Whether it was successful, meaning whether an entry existed for it.

HasItem

sub HasItem #(ExtensionID extension,
string itemString) => bool

Returns whether there is an item defined for the passed ExtensionID and item string.

GetAllItemsHashRef

sub GetAllItemsHashRef #(ExtensionID extension) => hashref

Returns a hashref of all the items defined for an extension.  Do not change the contents.  The keys are the item strings and the values are NaturalDocs::SourceDB::Items or derived classes.

Definition Functions

AddDefinition

sub AddDefinition #(
   ExtensionID extension,
   string itemString,
   FileName file,
   optional NaturalDocs::SourceDB::ItemDefinition definition
) => bool

Adds a definition to an item.  Assumes the item was already created with AddItem().  If there’s already a definition for this file in the item, the new definition will be ignored.

Parameters

extensionThe ExtensionID.
itemStringThe item string.
fileThe FileName the definition is in.
definitionIf you’re using a custom NaturalDocs::SourceDB::ItemDefinition class, you must include an object for it here.  Otherwise this parameter is ignored.

Returns

Whether the definition was added, which is to say, whether this was the first definition for the passed FileName.

ChangeDefinition

sub ChangeDefinition #(ExtensionID extension,
string itemString,
FileName file,
NaturalDocs::SourceDB::ItemDefinition definition)

Changes the definition of an item.  This function is only used for extensions that use custom NaturalDocs::SourceDB::ItemDefinition-derived classes.

Parameters

extensionThe ExtensionID.
itemStringThe item string.
fileThe FileName the definition is in.
definitionThe definition, which must be an object derived from NaturalDocs::SourceDB::ItemDefinition.

GetDefinition

sub GetDefinition #(
   ExtensionID extension,
   string itemString,
   FileName file
) => NaturalDocs::SourceDB::ItemDefinition or bool

If the extension uses custom NaturalDocs::SourceDB::ItemDefinition classes, returns it for the passed definition or undef if it doesn’t exist.  Otherwise returns whether it exists.

DeleteDefinition

sub DeleteDefinition #(ExtensionID extension,
string itemString,
FileName file) => bool

Removes the definition for the passed item.  Returns whether it was successful, meaning whether a definition existed for that file.

HasDefinitions

sub HasDefinitions #(ExtensionID extension,
string itemString) => bool

Returns whether there are any definitions for this item.

HasDefinition

sub HasDefinition #(ExtensionID extension,
string itemString,
FileName file) => bool

Returns whether there is a definition for the passed FileName.

Watched File Functions

WatchFileForChanges

sub WatchFileForChanges #(FileName filename)

Begins watching a file for changes.  Only one file at a time can be watched.

This should be called before a file is parsed so the file info goes both into the main database and the watched file info.  Afterwards you call AnalyzeWatchedFileChanges() so item deletions and definition changes can be detected.

Parameters

filenameThe FileName to watch.

WatchingFileForChanges

sub WatchingFileForChanges # => bool

Returns whether we’re currently watching a file for changes or not.

AnalyzeWatchedFileChanges

sub AnalyzeWatchedFileChanges

Analyzes the watched file for changes.  Will delete and change definitions as necessary.

A package that handles all the gory details of managing symbols.
SourceDB is an experimental package meant to unify the tracking of various elements in the source code.
sub RegisterExtension #(package extension,
bool usesDefinitionObjects) => ExtensionID
Registers a NaturalDocs::SourceDB::Extension-derived package and returns a unique ExtensionID for it.
my @extensions
An array of NaturalDocs::SourceDB::Extension-derived extensions, as added with RegisterExtension().
A base package for all SourceDB extensions.
my @extensionUsesDefinitionObjects
An array where the indexes are ExtensionIDs and the values are whether that extension uses its own definition class derived from NaturalDocs::SourceDB::ItemDefinition or it just tracks their existence.
A unique identifier for each NaturalDocs::SourceDB extension as given out by RegisterExtension().
A base class for all item definitions for extensions that track more than existence.
my @items
The array of source items.
my %files
A hashref mapping source FileNames to NaturalDocs::SourceDB::Files.
A string representing the absolute, platform-dependent path to a file.
A class used to index items by file.
my $watchedFile
When a file is being watched for changes, will be a NaturalDocs::SourceDB::File for that file.
my $watchedFileName
When a file is being watched for changes, will be the FileName of the file being watched.
my $watchedFileDefinitions
When a file is being watched for changes, will be a NaturalDocs::SourceDB::WatchedFileDefinitions object.
A class to track the definitions appearing in a watched file.
sub Load
Loads the data of the source database and all the extensions.
sub Save
Saves the data of the source database and all its extensions.
sub PurgeDeletedSourceFiles
Removes all data associated with deleted source files.
sub AddItem #(ExtensionID extension,
string itemString,
NaturalDocs::SourceDB::Item item) => bool
Adds the passed item to the database.
sub GetItem #(ExtensionID extension,
string itemString) => bool
Returns the NaturalDocs::SourceDB::Item-derived object for the passed ExtensionID and item string, or undef if there is none.
A base class for something being tracked in NaturalDocs::SourceDB.
sub DeleteItem #(ExtensionID extension,
string itemString) => bool
Deletes the record of the passed ExtensionID and item string.
sub HasItem #(ExtensionID extension,
string itemString) => bool
Returns whether there is an item defined for the passed ExtensionID and item string.
sub GetAllItemsHashRef #(ExtensionID extension) => hashref
Returns a hashref of all the items defined for an extension.
sub AddDefinition #(
   ExtensionID extension,
   string itemString,
   FileName file,
   optional NaturalDocs::SourceDB::ItemDefinition definition
) => bool
Adds a definition to an item.
sub ChangeDefinition #(ExtensionID extension,
string itemString,
FileName file,
NaturalDocs::SourceDB::ItemDefinition definition)
Changes the definition of an item.
sub GetDefinition #(
   ExtensionID extension,
   string itemString,
   FileName file
) => NaturalDocs::SourceDB::ItemDefinition or bool
If the extension uses custom NaturalDocs::SourceDB::ItemDefinition classes, returns it for the passed definition or undef if it doesn’t exist.
sub DeleteDefinition #(ExtensionID extension,
string itemString,
FileName file) => bool
Removes the definition for the passed item.
sub HasDefinitions #(ExtensionID extension,
string itemString) => bool
Returns whether there are any definitions for this item.
sub HasDefinition #(ExtensionID extension,
string itemString,
FileName file) => bool
Returns whether there is a definition for the passed FileName.
sub WatchFileForChanges #(FileName filename)
Begins watching a file for changes.
sub WatchingFileForChanges # => bool
Returns whether we’re currently watching a file for changes or not.
sub AnalyzeWatchedFileChanges
Analyzes the watched file for changes.
The topmost style containing everything.
A NaturalDocs::SourceDB-based package that manages all the image references appearing in source files.
sub Load # => bool
Called by NaturalDocs::SourceDB->Load() to load the extension’s data.
sub Save
Called by NaturalDocs::SourceDB->Save() to save the extension’s data.
Close