SourceDB is an experimental package meant to unify the tracking of various elements in the source code.
| NaturalDocs:: | SourceDB is an experimental package meant to unify the tracking of various elements in the source code. |
| The Idea | For quite a while Natural Docs only needed SymbolTable. |
| Assumptions | SourceDB is built around certain assumptions. |
| Types | |
| ExtensionID | A unique identifier for each NaturalDocs::SourceDB extension as given out by RegisterExtension(). |
| Variables | |
| extensions | An array of NaturalDocs::SourceDB::Extension-derived extensions, as added with RegisterExtension(). |
| extensionUsesDefinitionObjects | An array where the indexes are ExtensionIDs and the values are whether that extension uses its own definition class derived from NaturalDocs::SourceDB::ItemDefinition or it just tracks their existence. |
| items | The array of source items. |
| files | A hashref mapping source FileNames to NaturalDocs::SourceDB::Files. |
| watchedFile | When a file is being watched for changes, will be a NaturalDocs::SourceDB::File for that file. |
| watchedFileName | When a file is being watched for changes, will be the FileName of the file being watched. |
| watchedFileDefinitions | When a file is being watched for changes, will be a NaturalDocs::SourceDB::WatchedFileDefinitions object. |
| Extension Functions | |
| RegisterExtension | Registers a NaturalDocs::SourceDB::Extension-derived package and returns a unique ExtensionID for it. |
| File Functions | |
| Load | Loads the data of the source database and all the extensions. |
| Save | Saves the data of the source database and all its extensions. |
| PurgeDeletedSourceFiles | Removes all data associated with deleted source files. |
| Item Functions | |
| AddItem | Adds the passed item to the database. |
| GetItem | Returns the NaturalDocs::SourceDB::Item-derived object for the passed ExtensionID and item string, or undef if there is none. |
| DeleteItem | Deletes the record of the passed ExtensionID and item string. |
| HasItem | Returns whether there is an item defined for the passed ExtensionID and item string. |
| GetAllItemsHashRef | Returns a hashref of all the items defined for an extension. |
| Definition Functions | |
| AddDefinition | Adds a definition to an item. |
| ChangeDefinition | Changes the definition of an item. |
| GetDefinition | If the extension uses custom NaturalDocs::SourceDB::ItemDefinition classes, returns it for the passed definition or undef if it doesn’t exist. |
| DeleteDefinition | Removes the definition for the passed item. |
| HasDefinitions | Returns whether there are any definitions for this item. |
| HasDefinition | Returns whether there is a definition for the passed FileName. |
| Watched File Functions | |
| WatchFileForChanges | Begins watching a file for changes. |
| WatchingFileForChanges | Returns whether we’re currently watching a file for changes or not. |
| AnalyzeWatchedFileChanges | Analyzes the watched file for changes. |
For quite a while Natural Docs only needed SymbolTable. However, 1.3 introduced the ClassHierarchy package which duplicated some of its functionality to track classes and parent references. 1.4 now needs ImageReferenceTable, so this package was an attempt to isolate the common functionality so the wheel doesn’t have to keep being rewritten as the scope of Natural Docs expands.
SourceDB is designed around Extensions and items. The purposefully vague “items” are anything in the source code that we need to track the definitions of. Extensions are the packages to track them, only they’re derived from NaturalDocs::SourceDB::Extension and registered with this package instead of being free standing and duplicating functionality such as watched files.
The architecture on this package isn’t comprehensive yet. As more extensions are added or previously made free standing packages are migrated to it it will expand to encompass them. However, it’s still experimental so this concept may eventually be abandoned for something better instead.
SourceDB is built around certain assumptions.
SourceDB assumes that only the first item per file with a particular item string is relevant. For example, if two functions have the exact same name, there’s no way to link to the second one either in HTML or internally so it doesn’t matter for our purposes. Likewise, if two references are exactly the same they go to the same target, so it doesn’t matter whether there’s one or two or a thousand. All that matters is that at least one reference exists in this file because you only need to determine whether the entire file gets rebuilt. If two items are different in some meaningful way, they should generate different item strings.
SourceDB assumes the parse method is that the information that was stored from Natural Docs’ previous run is loaded, a file is watched, that file is reparsed, and then AnalyzeWatchedFileChanges() is called. When the file is reparsed all items within it are added the same as if the file was never parsed before.
If there’s a new item this time around, that’s fine no matter what. However, a changed item wouldn’t normally be recorded because the previous run’s definition is seen as the first one and subsequent ones are ignored. Also, deleted items would normally not be recorded either because we’re only adding.
The watched file method fixes this because everything is also added to a second, clean database specifically for the watched file. Because it starts clean, it always gets the first definition from the current parse which can then be compared to the original by AnalyzeWatchedFileChanges(). Because it starts clean you can also compare it to the main database to see if anything was deleted, because it would appear in the main database but not the watched one.
This means that functions like ChangeDefinition() and DeleteDefinition() should only be called by AnalyzeWatchedFileChanges(). Externally only AddDefinition() should be called. DeleteItem() is okay to be called externally because entire items aren’t managed by the watched file database, only definitions.
A unique identifier for each NaturalDocs::SourceDB extension as given out by RegisterExtension().
my @extensions
An array of NaturalDocs::SourceDB::Extension-derived extensions, as added with RegisterExtension(). The indexes are the ExtensionIDs and the values are package references.
my @extensionUsesDefinitionObjects
An array where the indexes are ExtensionIDs and the values are whether that extension uses its own definition class derived from NaturalDocs::SourceDB::ItemDefinition or it just tracks their existence.
my @items
The array of source items. The ExtensionIDs are the indexes, and the values are hashrefs mapping the item string to NaturalDocs::SourceDB::Item-derived objects. Hashrefs may be undef.
my %files
A hashref mapping source FileNames to NaturalDocs::SourceDB::Files.
my $watchedFile
When a file is being watched for changes, will be a NaturalDocs::SourceDB::File for that file. Is undef otherwise.
When the file is parsed, items are added to both this and the version in files. Thus afterwards we can compare the two to see if any were deleted since the last time Natural Docs was run, because they would be in the files version but not this one.
my $watchedFileName
When a file is being watched for changes, will be the FileName of the file being watched. Is undef otherwise.
my $watchedFileDefinitions
When a file is being watched for changes, will be a NaturalDocs::SourceDB::WatchedFileDefinitions object. Is undef otherwise.
When the file is parsed, items are added to both this and the version in items. Since only the first definition is kept, this will always have the definition info from the file whereas the version in items will have the first definition as of the last time Natural Docs was run. Thus they can be compared to see if the definitions of items that existed the last time around have changed.
sub RegisterExtension #( package extension, bool usesDefinitionObjects ) => ExtensionID
Registers a NaturalDocs::SourceDB::Extension-derived package and returns a unique ExtensionID for it. All extensions must call this before they can be used.
The order in which extensions register is important. Whenever possible, items are added in the order their extensions registered. However, items are changed and deleted in the reverse order. Take advantage of this to minimize churn between extensions that are dependent on each other.
For example, when symbols are added or deleted they may cause references to be retargeted and thus their files need to be rebuilt. However, adding or deleting references never causes the symbols’ files to be rebuilt. So it makes sense that symbols should be created before references, and that references should be deleted before symbols.
| extension | The package or object of the extension. Must be derived from NaturalDocs::SourceDB::Extension. |
| usesDefinitionObjects | Whether the extension uses its own class derived from NaturalDocs::SourceDB::ItemDefinition or simply tracks each definitions existence. |
An ExtensionID unique to the extension. This should be saved because it’s required in functions such as AddItem().
sub Load
Loads the data of the source database and all the extensions. Will call NaturalDocs::SourceDB::Extension->Load() for all of them, unless there’s a situation where all the source files are going to be reparsed anyway in which case it’s not needed.
sub Save
Saves the data of the source database and all its extensions. Will call NaturalDocs::SourceDB::Extension->Save() for all of them.
sub AddItem #( ExtensionID extension, string itemString, NaturalDocs::SourceDB:: Item item ) => bool
Adds the passed item to the database. This will not work if the item string already exists. The item added should not already have definitions attached. Only use this to add blank items and then call AddDefinition() instead.
| extension | An ExtensionID. |
| itemString | The string serving as the item identifier. |
| item | An object derived from NaturalDocs::SourceDB::Item. |
Whether the item was added, that is, whether it was the first time this item was added.
sub GetItem #( ExtensionID extension, string itemString ) => bool
Returns the NaturalDocs::SourceDB::Item-derived object for the passed ExtensionID and item string, or undef if there is none.
sub DeleteItem #( ExtensionID extension, string itemString ) => bool
Deletes the record of the passed ExtensionID and item string. Do not delete items that still have definitions. Use DeleteDefinition() first.
| extension | The ExtensionID. |
| itemString | The item’s identifying string. |
Whether it was successful, meaning whether an entry existed for it.
sub HasItem #( ExtensionID extension, string itemString ) => bool
Returns whether there is an item defined for the passed ExtensionID and item string.
sub GetAllItemsHashRef #( ExtensionID extension ) => hashref
Returns a hashref of all the items defined for an extension. Do not change the contents. The keys are the item strings and the values are NaturalDocs::SourceDB::Items or derived classes.
sub AddDefinition #( ExtensionID extension, string itemString, FileName file, optional NaturalDocs::SourceDB:: ItemDefinition definition ) => bool
Adds a definition to an item. Assumes the item was already created with AddItem(). If there’s already a definition for this file in the item, the new definition will be ignored.
| extension | The ExtensionID. |
| itemString | The item string. |
| file | The FileName the definition is in. |
| definition | If you’re using a custom NaturalDocs::SourceDB::ItemDefinition class, you must include an object for it here. Otherwise this parameter is ignored. |
Whether the definition was added, which is to say, whether this was the first definition for the passed FileName.
sub ChangeDefinition #( ExtensionID extension, string itemString, FileName file, NaturalDocs::SourceDB:: ItemDefinition definition )
Changes the definition of an item. This function is only used for extensions that use custom NaturalDocs::SourceDB::ItemDefinition-derived classes.
| extension | The ExtensionID. |
| itemString | The item string. |
| file | The FileName the definition is in. |
| definition | The definition, which must be an object derived from NaturalDocs::SourceDB::ItemDefinition. |
sub GetDefinition #( ExtensionID extension, string itemString, FileName file ) => NaturalDocs::SourceDB::ItemDefinition or bool
If the extension uses custom NaturalDocs::SourceDB::ItemDefinition classes, returns it for the passed definition or undef if it doesn’t exist. Otherwise returns whether it exists.
sub HasDefinition #( ExtensionID extension, string itemString, FileName file ) => bool
Returns whether there is a definition for the passed FileName.
sub WatchFileForChanges #( FileName filename )
Begins watching a file for changes. Only one file at a time can be watched.
This should be called before a file is parsed so the file info goes both into the main database and the watched file info. Afterwards you call AnalyzeWatchedFileChanges() so item deletions and definition changes can be detected.
| filename | The FileName to watch. |
Registers a NaturalDocs::SourceDB::Extension-derived package and returns a unique ExtensionID for it.
sub RegisterExtension #( package extension, bool usesDefinitionObjects ) => ExtensionID
An array of NaturalDocs::SourceDB::Extension-derived extensions, as added with RegisterExtension().
my @extensions
An array where the indexes are ExtensionIDs and the values are whether that extension uses its own definition class derived from NaturalDocs::SourceDB::ItemDefinition or it just tracks their existence.
my @extensionUsesDefinitionObjects
The array of source items.
my @items
A hashref mapping source FileNames to NaturalDocs::SourceDB::Files.
my %files
When a file is being watched for changes, will be a NaturalDocs::SourceDB::File for that file.
my $watchedFile
When a file is being watched for changes, will be the FileName of the file being watched.
my $watchedFileName
When a file is being watched for changes, will be a NaturalDocs::SourceDB::WatchedFileDefinitions object.
my $watchedFileDefinitions
Loads the data of the source database and all the extensions.
sub Load
Saves the data of the source database and all its extensions.
sub Save
Removes all data associated with deleted source files.
sub PurgeDeletedSourceFiles
Adds the passed item to the database.
sub AddItem #( ExtensionID extension, string itemString, NaturalDocs::SourceDB:: Item item ) => bool
Returns the NaturalDocs::SourceDB::Item-derived object for the passed ExtensionID and item string, or undef if there is none.
sub GetItem #( ExtensionID extension, string itemString ) => bool
Deletes the record of the passed ExtensionID and item string.
sub DeleteItem #( ExtensionID extension, string itemString ) => bool
Returns whether there is an item defined for the passed ExtensionID and item string.
sub HasItem #( ExtensionID extension, string itemString ) => bool
Returns a hashref of all the items defined for an extension.
sub GetAllItemsHashRef #( ExtensionID extension ) => hashref
Adds a definition to an item.
sub AddDefinition #( ExtensionID extension, string itemString, FileName file, optional NaturalDocs::SourceDB:: ItemDefinition definition ) => bool
Changes the definition of an item.
sub ChangeDefinition #( ExtensionID extension, string itemString, FileName file, NaturalDocs::SourceDB:: ItemDefinition definition )
If the extension uses custom NaturalDocs::SourceDB::ItemDefinition classes, returns it for the passed definition or undef if it doesn’t exist.
sub GetDefinition #( ExtensionID extension, string itemString, FileName file ) => NaturalDocs::SourceDB::ItemDefinition or bool
Removes the definition for the passed item.
sub DeleteDefinition #( ExtensionID extension, string itemString, FileName file ) => bool
Returns whether there are any definitions for this item.
sub HasDefinitions #( ExtensionID extension, string itemString ) => bool
Returns whether there is a definition for the passed FileName.
sub HasDefinition #( ExtensionID extension, string itemString, FileName file ) => bool
Begins watching a file for changes.
sub WatchFileForChanges #( FileName filename )
Returns whether we’re currently watching a file for changes or not.
sub WatchingFileForChanges # => bool
Analyzes the watched file for changes.
sub AnalyzeWatchedFileChanges
Called by NaturalDocs::SourceDB->Load() to load the extension’s data.
sub Load # => bool
Called by NaturalDocs::SourceDB->Save() to save the extension’s data.
sub Save