CPP Unit is a C++ unit testing framework. We are using this framework for automatic testing of kernel interface and its functionality. All test cases are placed in kernel/tests directory and they are linked to kernel/kernel_tests binary output. We have implemented test classes for all interface objects. Each class is specialized for certain class interface object. Each class has general name form

	TestClassName

where ClassName stands for tested class. Test class implements test suite which is identified by its name. Main test program runs all test suites specified by name (the section called “Test program”). Each test suit consists of test cases which test particular behavior of tested class.

	class TestClassName : public CppUnit::TestFixture
	{
		// defines this class as test suite
		CPPUNIT_TEST_SUITE(TestClassName);
			// This method has to be implemented as test case
			CPPUNIT_TEST(TestMethod);

			// definition of other test cases
		CPPUNIT_TEST_SUITE_END();
	public:
		void setUp()
		{
			// this method should initialize local test class
			// data used in tests
		}

		void tearDown()
		{
			// clean up after setUp method
		}

		void Test()
		{
			// Implementation of testcase
			CPPUNIT_ASSERT(expected==op())

			// ...

			// sometimes we need to test that something throws an
			// exception
			try
			{
				// this operation should throw with this parameters
				op(parameters);

				// exception hasn't occured, we will force failure
				CPPUNIT_FAIL("This operation should have failed.");
			}catch(ExceptionType & e)
			{
				// ok, exception has been thrown
			}
		}
		
	};

	// registers this class to CPP Unit framework and assigns it
	// with given name
	CPPUNIT_TEST_SUITE_REGISTRATION(TestClassName);
	CPPUNIT_TEST_SUITE_NAMED_REGISTRATION(TestClassName, "TEST_CLASSNAME");

Kernel, as the lowest layer, is responsible for maintaing of pdf content from file and to provide object interface for making changes to higher layer. We will call this objects as cobjects. More precisely - highlevel cobjects (CPdf, CPage, etc.) which provide higher pdf entities logic and lowlevel cobjects which are pdf data types carrier (CInt, CArray, CDict, CString, etc.). Values stored in lowlevel cobjects are also called properties and they are wrapped by IProperty class. Properties are identified by indirect reference (the way how pdf adresses entities). User of kernel should start with CPdf instance which provides all properties from document as well as access to document pages or outlines. Pages then provide access to Annotations. All cobjects are returned wrapped by shared_ptr (see Chapter 1, Used technologies).

Kernel uses Xpdf code for document content parsing. XPdf's XRef class provides fetching and parsing functionality. Oposite way (from cobjects to file writing) is provided by IPdfWriter implementation. XPdf Stream class is replaced by StreamWriter kernel class.

CXRef class inherits from XRef (xpdf class) and adds internals for storing of changed objects not public for direct user. XRefWriter enables interface for making changes inherited from CXRef. See the section called “3 layer model” for more information.

Kernel architecture

Scripting architecture

Scripting is base of the editor functionality. Each editor window have its own script context and scripts run independently in them. On creating of each window, the scripting base is constructed (BaseGUI, extended by GUI specific functions from Base). The Base will construct some necessary objects:

Window will set ConsoleWriter object, that will handle script output. There are two classes derived from ConsoleWriter:

Class Base (respectively BaseGui or BaseConsole) export many functions as slots. These are visible as static functions in the script and they are main way of communication between the . BaseCore class, from which the Base class is extended does not provide any static functions, but it provides basic script functionality - garbage collection, support for callback functions, running init scripts and handling of script errors.

Wrappers.  Script need to manipulate with objects in PDf and in editor. Due to the limitations of QSA, every C++ object (except some basic types, such as strings, numbers and some QT types, such as QColor) need to be derived from QObject to be usable in scripting and only functions exported as slots will be available for script. Due to this limitation, wrappers need to exist around most objects, like tree items in object tree (QSTreeItem, QSTreeItemContentStream), PDF objects (QSAnnotation, QSArray , QSContentStream, QSDict , QSIProperty, QSPage , QSPdf, QSPdfOperator and QSStream), class for invoking popup menus (QSMenu) and helper classes related to PDF objects (QSIPropertyarray ,QSPdfOperatorIterator and QSPdfOperatorStack). All wrappers are derived from QSCObject class, which provides some basic function for memory handling and error handling.

Source of script input. 

Scripting API documentation.  Description of static scripting functions, functions provided by settings and PageSpace objects and description of scripting objects and their methods is included in the user documentation.

Console mode.  Functionality in console mode is similar, with few exceptions:

Basic class in GUI is PdfEditorWindow, which represent the main editor window. Application can have more such windows opened, in each of them editing different document. On top of the window is menubar and toolbar (although being on top is only default position, user can move all toolbars as he wants, all toolbars will dock on either of four sides of the editor window, or they can float outside of it).

All toolbars are of ToolBar class, derived from QT class QToolBar. The menubar is standard QT QMenuBar, although filling the menubar and also toolbar with its items and maintaining them and their association to script code is responsility of class Menu. On bottom there is a statusbar, which can be used to show various information (class StatusBar, derived from QT class QStatusBar) Rest of the area not occupied by statusbar, menu and any toolbars is divided by movable splitter on left and right part. Left part is divided by another splitted into part with preview window (class PageSpace) on top and commandline window, providing script input and output (class CommandWindow) on bottom. Right part is also divided by splitter, on upper part there is object tree view (class MultiTreeWindow), on lower part is property editor (class PropertyEditor).

Every element mentioned above (except menu and the preview window) can he hidden and shown by user. Application will remember the element layout (size and position of window, position of splitters and position of toolbars) in settings when closing and will reopen next time with the same layout.

Dialogs are also part of the GUI. Many simple dialogs are handled by script, but as script is unable to create more complex dialogs, some of them had to be implemented directly in C++. They are:

Also, some standard system dialogs (to pick font, color or name of file) are used.

Delinearizator is class which provides simple interface for pdf document delinearization (see also Linearized pdf document). Class instances are created by factory method getInstance (see Factory method design pattern) and one instance handles one pdf file, which has to be linearized. If file is not linearezed, instance is not created and exception is thrown. When instance is created, document can be simply delinearized by delinearize method.

As well as XRefWriter it also uses IPdfWriter impementator for content writing (this can be changed in runtime and provides flexibility for output format).

Delinearizator itslef is build on top of Xpdf XRef class which provides object fetching functionality and cross reference table maintainance. This is used for fetching of all objects (without those which are specific for linearized content) and IPdfWriter implementation is used to write them to the new file.

This class (implemented in src/utils/delinearizator.cc) depends on xpdf code and kernel/pdfwriter module.

Delinearizator architecture

IConfigurationParser provides interface for underlaying stream parsing where stream data can be somehow (depends on format and parser implementation) transformed to key, value pairs, where key stands for data identifier and value is associated with this key.

Class is template and abstract which means that implementators have to implement all methods and supply data types for key and value. IConfigurationParser is defined in src/utils/confparser.h file. We have implemented simple implementation in StringConfigurationParser class which parses file with simple format:

	# comments are ignore
	% this allso stand for comment by default
	key : value # this key value is associaed with value

Where both key and value are strings. This parser can be configured to ignore comments (strings starting with character from commentsSet), to use different delimiter character (the one which separates key from value) by setting delimiterSet or to set characters which should be considered as blank characters (by setting blankSet).

Configuration parser code doesn't depend on pdfedit or xpdf code and can be reused as it is without any changes. It uses STL streams. In PDFedit project it is used e. g. in ModeController or OperatorHinter classes.

Configuration parser object structure.

RulesManager is simple concept based on association of rule and its target. Implementation uses C++ template mechanism to be generic in way of data type definition for both rule and target. Both types have to fullfill certain contracts (see doxygen program documentation for more details). Rules are keys in internal storage and they are associated with their targets (1:1 relation).

Described storage with data forms RulesManager configuration. Second part is based on IRuleMatcher (implementator of this abstract class). It has responsibility to provide logic related to rules choosing, evaluating of compatibility of rules and defining priority for rules. When findMatching method is called, matcher is considered to choose association from storage which matches given rule the best. Implementator of matcher has to implement class Functor so that it describes when given rule matches given original rule and also provide with priority of this match.

Class user just defines rule and target data types, implements IRuleMatcher for rule data type with matching logic and use class as it is. RulesManager also enables loading rule, value configuration from configuration file. loadFromFile uses Parser template data type. IConfigurationParser implementator with rule type for key and target type for value can be used here.

This concept was used for ModeController class and OperatorHinter in our project. (the section called “ModeController”).

Rules manager object structure

IObserver class which stands for observer in following context is mechanism to allow announcing internal object state change to other objects. Object with internal state which announces is observer handler and classes which monitor (observe) are called observers. This is basic idea of Observer design patter. This implementation keeps basic contracts of this pattern and adds additional functionality to be as flexible as possible.

From class point of view observer handler is class which inherits from ObserverHandler class. This template class provides interface for observer registration and unregistration. Each observer has to be registered before it is notified about changes. When it is no longer interested in changes, it should unregister itself from observed objects. ObserverHandler also provides method which announces all registered observer about change (notifyObservers method). Observers are called in order which depends on their priorities and for same priorities on registration order. Observer handler is responsible to call this notifyObservers method whenever its internal state has changed (and he wants to announce this change) and provides correct parameters for it.

Observer has to implement IObserver abstract class. The most important to implement is notify method. This method is called by observer handler after its state has changed. Observer can use given newValue parameter which holds current value which has changed. Additional information can be obtained from given context (see bellow). notify method can use given values to update its internal state or to do additional actions but in any case it shouldn't modify given data (this can lead to end less loop, because observer handler notifies about change and so observer is notified again and this will never stops).

Observer and observer handler cooperation

Notifying would be rather poor if just newValue was available. So our observer concept adds context to notify method. This context keeps additional information. Context hierarchy is based on IChangeContext abstract class. This provides just information about type of context. notify implementator should check this type and cast given context to correct type and use it as needed. If observer handler doesn't want to give any context, this should keep NULL value.

We have implemented following types of contexts because they were needed by project.

New context types can be defined as well and very easily - just inherit from IChangeContext and implement getType to return correct type enum value and add some information specific for context inside to class.

Observer concept classes

Note that all mentioned classes are C++ template classes and they use data type as template parameter. This type stands for data type of value which change is announced (newValue in notify method e. g.). All instances are wrapped by share_ptr (boost smart pointer) to prevent from data life cycle problems because share_ptr correctly shares data between multiple user and they are deallocated in moment when nobody holds shared pointer with them. If shared pointers are used correctly (this means that object wrapped inside is never deallocated by hand), no problems should occure with object instances. This is very important because obsever handler doesn't have any information whether observers, which are registered are alive in moment when it calls their notify method.

Second restriction for implementators and users is that notify method as well as all other methods (also constructors and desctructor) can't throw an exception. This is intention, because observer handler has to guarantee that each observer is called after it finishes notifyObservers method. It doesn't know anything about observers so it also doesn't know how to handle their exceptions. Only reaction would be (to keep contract that all observers are notified) to silently ignore (or to log) exception. This could lead to inconsistencies and so it is safer to forbit exceptions at all. notify method implementator has to keep this in mind because, if exception is thrown and it is forbiden in method signature (throw() clause after method definition), program is forced to terminate by default.

Iterator is a specific implementation of Iterator design pattern. It is used to traverse an arbitrary linked list that meets few requirements. Main goal of this iterator implementation is to be flexible and easily extensible because we need many special iterators iterating only over specific items.

The iterator is bidirectional. Information about previous and next item is obtained from the item itself. Sometimes it is not possible to have a container outside stored items, what would be more flexible, but the information must be stored in the items itself. Item before first item and item after last item are not valid objects.

New special iterator can be easily created from the base iterator just by inheriting and overloading one function which selects valid items. Example of special iterators can be found in the section called “Pdfoperator iterator”.

Iterator

Code of XPDF project couldn't have been reused without modifications because it is not prepared for making changes to its objects very well (code assumes data reading and using them - not to use and change them). Our code changes can be divided to 3 cathegories:

For more information see following detailed description.

	Object::clone()const;

	void initInternals(Guint pos);
	void destroyInternals();

	Guint eofPos;

	virtual RefState knowsRef(Ref ref);

	Array * clone()const;

	Dict * clone()const;

	Object * update(char * key, Object * val);	    

	Object * del(const char * key);	 

Stream.h and Stream.cc

Xpdf code defines Stream hierarchy to describe pdf stream objects. Streams (as pdf data types) define container of pdf objects. This container is associated with dictionary which contains information about its length and filters which were used for stream encoding. XRef class reads data from stream or Content stream object is based on stream.

Design changes

Stream is base class for both normal streams represented by BaseStream (Stream descendant) base class and FilterStream (also direct Stream descendant) base class used for all filered streams. This stream objects hierarchy is strictly specialized for reading and can't be used for making changes to stream data. CXref and XRefWriter however needs to make transparent modifications to stream with pdf content (so that xpdf code using Streams doesn't have to be changed very much). This is the reason for some changes in Stream hierarchy design.

Problem with stream modification is solved by new abstract class (base class for all specialized stream modificators) StreamWriter. This defines interface for stream writing (in same way as Stream defines operations for reading). However, implementation of concrete writer requires (such as FileStreamWriter) multiple inheritance, because it needs interface from StreamWriter and also access to concrete BaseStream (in FileStreamWriter it is FileStream) fields. So original inheritance of all direct descendants of Stream and BaseStream had to be changed to virtual (to prevent ambiguity). This model enables transparent usage of StreamWriter as Stream typed instances in xpdf code and as StreamWriter typed instances in our higher level classes (like FileStreamWriter) in pdfedit code for writing.

FilterStream hierarchy is untouched in design way, because our project doesn't change filtered streams directly. It works just with base stream, because FilterStream hierarchy is hard to be reused for encoding. So just decode functionality is used.

Stream hierarchy

New features

Stream object as one of complext value data type which is stored in Object (as all other data types) has to to provide cloning support. We have added abstract

	virtual Stream * clone()=0;	     

method in Stream base class. Each specific stream implementator has to provide its clone implementation. No default implementation is written in Stream directly to force all specific filters provide one. If any of filters is not able to create clone, this method should return NULL. This should not happen, however clone implementation has to be aware of it (and has to check whether filter stream has cloned correctly).

FileStream is stream which reads data directly from FILE stream and so cloning has to copy all data (from stream start to the end - if stream is limited, then just first length bytes) somewhere else. Creation of new file, just for temporarily created clone is not very effective and may produce several problems (not enough free place, creation and removing of temporary file, etc.). We have solved this problem by creating MemStream with buffer containing same data as FileStream. This brakes contract of clone meaning a bit, because cloned stream is not precisely the same as original, because it is represented by another Stream class. Nevertheless it keeps the most important contract, that user of Stream interface doesn't know the difference and clone and original don't affect each other when one is changed.

MemStream represent stream stored in buffer in the memory. So cloning is straightforward and just buffer is copied for new MemStream. All other attributes are set according copied buffer. Buffer copying starts from start field position and lenght bytes are copied. So final MemStream will contain just data used in original one. Finaly needFree field is always set to true, because we have allocated new buffer for clone and so it has to be deallocated when MemStream is destroyed. ^[4]

EmbedStream is special stream which is stored in some other stream. Clonig is also very simple, because this stream just holds one Stream pointer field and some attributes which doesn't change during instance life cycle. Cloning is then just delegation to cloning of stream field and creating new EmbedStream with cloned value and same parameters which were used for given instance.

FilterStream stream branch represents different types of filters which can be used to encode stream data (Pdf specification describes which filters can be used). Hierarchy and design of filters follows Decorator design patter and each filter works with underlaying stream (which is stored as pointer) field which is typed as Stream (so it can be either stream with data - MemStream, FileStream or EmbedStream - or another filter stream).

FileStream is cloned in similar way as EmbedStream. Each filter implemenetator holds Stream pointer (as already mentioned). This is cloned by clone method (defined in super type). When underlaying stream is cloned (and clone method returned wih non NULL value - which means that this stream supports clonning), current stream creates new same filter stream instance with same configuration parameters (these are usually parameters which were given to it in constructor - but when such attributes can change in time they have to be stored somewhere else in constructor specially for this purpose). ^[5] General implementation for all filter streams is as follows:

	// clones underlying stream
	Stream * cloneStream=str->clone();
	if(!cloneStream)
	// if underlying stream doesn't support cloning, it will fail too
	return NULL;

	// creates same typed filter stream with same parameters and cloned
	// stream  
	return new ...(cloneStream[, specific_parameters]);

As mentioned above, some filters are not able to reconstruct parameters given them as constructor parameters and so it is hard to reconstruct same filter. Specially all filters which holds StreamPredictor field has additional field with PredictorContext (added by us):

	struct PredictorContext
	{
	  int predictor;
	  int width;
	  int comps;
	  int bits;
	};

where all parameters needed for StreamPredictor creation are stored. This structure is initialized in constructor and never changed. It is just used for cloning.

Parser.h and Stream.cc

Xpdf reads pdf files using two level mechanism. The lowest level, called Lexer, decodes streams if necessary and reads byte after byte. The second level, called Parser, reads from Lexer byte after byte until one complete object is read. This can be applied recursively. Then it initializes xpdf Object class with type and data of the read object. Parser object can read objects either from a single stream or from an array of streams (simply reading one stream after another could result in incomplete objects returned at the end of a stream). Parser is used to parse all objects including content stream tags.

xpdf Parser class

New features

Content streams can consist of more streams. Decoding and then concatenating of these streams must form a valid content stream. The problem is, that the content stream can be split into streams at almost arbitrary place. Added feature which was missing in the Parser object is the information when it has finished reading one stream and started reading another. Added method

	// End of actual stream
	bool eofOfActualStream () const { return (1 == endOfActStream); }

where endOfActStream is new variable indicating how many objects have been buffered from current stream. Content streams can consist of many small valid content streams. When splitted correctly, user can easily delete/add small content streams. Changes made by PdfEditor can be considered as such small valid content streams. After saving our changes we want to see these changes separately to existing content streams. This new feature is used to split many streams (which create one page content stream) to many small content streams. Because of the object buffering done by Parser the new feature had to be implemented specially this way.

Pdf file consists of objects. These objects are referenced from a special structure forming a tree. Objects can be either simple (number, string, name, boolean value, reference, null) or complex (array, dictionary, stream).

CObject class hierarchy

General description

CObjects are objects in pdfedit which represent objects in pdf file. All cobjects are derived from one base class IProperty. Objects form a tree like structure so we can divide objects into single objects (leafs) and composite objects (nodes). This is an example of Composite design pattern. This is a different approach to xpdf implementation where each pdf object is represented by the same huge class. The concept of having exactly one class representing each pdf object leads to many problems:

• inheriting - unclear oo design, unmanagable, it breaks the idiom of one entity for one purpose

• adding changing operations - would result in even more monstrous class

• sometimes value inside class, sometimes outside - unclear oo design

There are many interesting design decisions in xpdf objects implementation. For example memory handling makes it almost unsound to delete objects from complex types. Memory allocation policy, that means who/when/how is to deallocate xpdf Object is a mess which could easily lead to either memory leaks or memory corruption. The new design counted with new object for each different pdf object. Because of the pdf decoding complexity (pdf can be encoded using many filters) these objects use xpdf Object just for initializing and dispatching changes to CXref object. Objects can not be simply copied, because it is not clear if a copy is a deep copy with all indirect object copied or not.

Object class hierarchy

Base class

Every object is derived from one base class - IProperty. This base class is a hanger which can be used to access child objects uniformly. This class is a read only interface for all properties. Objects can be created uninitialized or can be initialized from an xpdf Object of the same type and simple objects cat be initialized directly from a value.

Simple objects

Simple objects are very similar. They share behaviour and because of this also method names (only value keeper is different). They are represented by one class using c++ templates. One template class (CObjectSimple), parameterized by object type, represents all 7 types of simple objects.

Complex objects

It is more difficult with complex types. Each complex type must contain references to its children which are also pdf objects. A design decision was made to use smart pointers for referencing child objects. The reasons are:

1. Allocation and deallocation policy - we cannot be sure when an object is deallocated nobody holds a pointer to the object. This could be solved by implementing reference counting, but why reimplement the wheel.

2. Automatic deallocating when not needed.

Pdf objects can be referenced using ids which are similar to pointers. This brings many problems. One of them is the impossibility to delete such objects. Many of the problems are automatically handled by smart pointers.

Array and dictionary

CArray stores its children in a simple container indexed by position. CDict stores its children in container indexed by string which uniquely identifies a property. The beauty of smart pointers arise when deallocating an array, it automatically deallocates its children when not referenced and this is done recursively.

Streams

Streams are the most complicated from all pdf objects. The problem is that xpdf can decode pdf files but it can not do it the other way round. (it is because it never needs it) Xpdf impelementation of streams is very rough. We use boost filtering iostream which provide us with the necessary general concept of encoded streams. However we have to implement the filters ourselves. (the easiest way is to save decoded streams without any filters) We do not know the filters as long as we do not change the object. We can modify either raw encoded stream or we can save decoded stream which is automatically encoded when saved using avaliable filters. Each stream consists of a dictionary and stream buffer. The dictionary can not be accessed directly. Dictionary interface is simulated by simple methods which delegate the calls to the dictionary object. Buffer is stored in encoded form allowing us to return the same byte representation of an unchanged object as read from a pdf file. At the time of writing this not all reversed filters have been implemented.

Accessing streams

We access streams using Adapter design pattern implementing open/close interface. We need to be able to read from more streams byte per byte because content streams can be splitted anywhere. We decided to return only xpdf objects.

CPdf class is main pdf document class. It maintains document content using XRefWriter field, document catalog PDF dictionary and provides other specialized highlevel objects such as CPage (see the section called “CPage”) and COutline.

Main responsibility is to keep all objects (it provides) synchronized with data which are used for them. As an example, it has to keep CPage instances synchronized with current state of page tree.

In design terminology, CPdf provides Facade design pattern; to manipulate with file in document scope of view. All internal objects used for particular purposes are hidden from class user and CPdf provides interface for manipulation with it (as an example, CPdf uses XRefWriter (see the section called “XRefWriter”) which enables making changes to document, but exports only CXref (see the section called “CXRef”) which enables just objects fetching - almost same interface as Xpdf XRef class).

Provided high level objects

CPdf provides high level objects maintaining some specialized part of document Pdf document catalog. These objects brings logic on properties with special meaning in pdf document. ^[7]

CPdf facade scheme

Pdf dictionaries referenced from Pdf page tree are called page objects. These dictionaries must contain a "Type" entry and the value must be a name object equal to "Page". Page objects are basic building blocks of pdf files. Each page can be independent with all required information stored in its dictionary. Some of its properties can be inherited from parent pages. Page object describes the appereance of the page (witdth, length, fonts used, rotation etc.).

The core of a page is one or more content streams which specify what is on a page (text, pictures, graphics ...).

Annotation is interactive entity which is associated with rectangle on page. They are organized in Annots array entry in page dictionary and so CPage is responsible for returning of all available annotations and also to provide interface to add new annotation.

Each annotations is described by dictionary, which has to contain at least Type element with Annot and Subtype element with concrete annotation type. Pdf specification describes several types of annotation types (e. g. text annotation - which describes text box floating upon normal page text, link annotation - which enables to jump to the target within document or to perform certain action when link is activated by mouse click). Rect element should be present as well, because it specifies where annotation should be spreaded.

CAnnotation represents such annotation and provides simple interface to manipulate elements in annotation dictionary. It implements ObserverHandler (see the section called “Observers”) to anable user to be informed about changes inside annotation dictionary. This class provides just simple interface for internal manipulation and it is intended to be base class for specific annotation types (no such specialized class are available yet, because they are not required by project in this moment).

	
	static boost::shared_ptr<CAnnotation> 
		createAnnotation(Rectangle rect, std::string annotType);
	

	
	bool registerInitializer(std::string annotType, boost::shared_ptr<IAnnotInitializator> impl, bool forceNew=false);
	

CAnnotation class hierarchy

CXRef class wrapps Xpdf XRef class and provides additional functionality with same interface (see Wrapper design patter. It provides with protected interface for making changes and internally stores changed objects. When object should be fetched (fetch method is called), it will check whether this object is already changed and if so, uses changed value. Otherwise delegates to XRef original implementation (this logic is kept in all methods defined in XRef).

CXRef inherits from XRef and so can be polymorphicaly used in xpdf code and this code doesn't need any changes to use CXref functionality. Aditional interface enables changes, but as we want to keep this making changes under control so it is protected and so accessible only for its inheritance descendants.

Added functionality includes:

For more information about CXref usage, please see Chapter 7, Changes to pdf document.

CXref class scheme

Content stream can consist of one or more pdf stream. It is responsible for everything on a page. If anything visible is changed the content stream must be changed. Content stream is a stream processed sequentially. Page can consist of one or more content streams and these streams must be concatenated before reading (objects can be splitted between two content streams). Content stream consists of operators and their operands. Each operator updates graphical state.

Storing pdf operators

Operators are processed sequentially and there are many situations when only some types of operators are needed. Clear solution is to use the Iterator design pattern. With this patter we can process operators one by one. If we need specific iterators we just create another child of basic iterator. There are be simple and composite operators. Operators form a tree-like structure. This is more readable than a list of all operators. So we implement another tree-like queue. Only the first level of operators is stored in ccontentstream. Each composite operator stores its children. This is an example of Composite design pattern. Simple and composite operators are accessed uniformly.

Operator queues

Every page has its content stream which contains the description of every object on a page. Content stream consists of operators and their operands specifying how to alter graphical state of a page. These operators specify what and how it is displayed. They are processed sequentially. If no content stream is avaliable (or is empty) the approperiate page is empty.

PdfOperators

Content stream consists of operators and their operands. Operators can be either simple or composite objects. The requirement for processing operators sequentially and representing operators in human readable form resulted in storing each operator in two queues.

Operator overview

Changing pdf operators

Changing the visible object properties means wrap those objects into other objects. Object can depend on previous objects. We need to be able to iterate backwards. Summarizing these requirements with human readable representation of operators leads to these decisions:

• sequentially processing the list, the order has to be the same as in the file we edit - Iterator design pattern

• grouping and wrapping operators - Composite design pattern and specific Decorator design patter

The first requirement is needed when simulating the display process of an pdf viewer (e.g. when finding out the absolute position of an operator). Composite design pattern used in representation of pdf operators is very useful, when changing objects (e.g. changing an existing text object to composite is very easy allowing us to add some special formatting). This becomes very useful with the Decorator design patter which allows us to change this object in conjuction with previous changes.

Example of changed pdf operator

Iterator design pattern allows us to iterate over items. When designed correctly specific iterator can be added easily.

Extending iterator

Basic iterator class implements one Template method. Specific iterators are created by overloading this method. It decides whether an item is valid or not. There are two important children of the base iterator class.

• AcceptingPdfOperatorIterator class

• RejectingPdfOperatorIterator class

These two classes either accept a set or operators or accept everything except those iterators. Extending these classes requires only defining the set of operators.

Pdf operator iterators overview

IPdfWriter abstract class provides interface for pdf content writing.

IPdfWriter writes pdf content in two phases:

Different implementator of IPdfWriter interface can be set by XRefWriter::setPdfWriter method.

Each sequence of writeContent, [writeContent, ]* writeTrailer forms new revision of document in incremental update sense. In this moment OldStylePdfWriter implementation is used which forms old style cross reference table (see pdf specification 2.4.3 Cross reference table chapter).

XRefWriter class inherits from CXref and provides

XRefWriter responsibility is to enable writing changes to the XRef's stream to make them visible after all required changes are finished. To separate from concrete implementation of storing, XRefWriter uses IPdfWriter abstract interface for object writing. Concrete implementation can be set in runtime and OldStylePdfWriter is used by default. XRefWriter is just responsible to provide with all changed objects which are retrieved from lower (CXref) layer (mapping is protected).

Underlaying stream with data which is stored in XRef supertype is typed as a Stream (see the section called “Stream.h and Stream.cc”. IPdfWriter requires StreamWriter (see the section called “Changes needed for code reuse”). So XRefWriter uses StreamWriter implementator in constructor and XRef use this stream when fetching objects and IPdfWriter uses it when writing content.

Finally XRefWriter keeps information needed for revision handling. ^[8] XRefWriter keeps an array of all available revisions. Index stands for revision number (0 is the newest one) and value is stream offset of associated cross reference section start for revision. This is used to get information about revision - for example to get revision size (see XRefWriter::getRevisionSize method) - and to enable revision changing. CXref implements

	void CXref::reopen(size_t xrefOff);

method which is responsible for clear XRef state change to forget all objects from current revision and to start parsing of cross reference section from given position. This enables to (rather simply) change current revision and so see documents in those state.

PDF format is prepared for such revisions and multi version documents very well, but doesn't support any kind of branching which means that changes can be done only for the newest revision. XRefWriter take in mind also this aspect and so all methods take care of current revision. This means that anytime current revision is not the newest one, changed objects are ignored and everything is delegated directly to the lowest layer. Also all methods producing changes throws ReadOnlyDocumentException. Even more all methods reimplemented by CXRef class, which depedns on changes, are omited and directly XRef implementation is used instead.

Pdf streams can be encoded using one or more filters specified by pdf specification. The encoding/decoding algorithm is called a filter. There are two types of filters

There is a set of filters which every pdf viewer must implement. It is up to the viewer how it handles an unknown filter.

Xpdf filters

Xpdf viewer implements all decoding filters from pdf specification v1.5. However it does not implement any encoding filters (it does not need them).

Simple xpdf stream hierarchy

Xpdf stream design is good but can be improved. The design resulted into many objects that are tightly coupled together (see picture). They can be decoupled using Facade design pattern which also makes objects more reusable. Finally, the unclear implementation makes it difficult to use and very difficult to extend and it lacks almost any fault tolerance. We use xpdf filters (because it is hardwired into xpdf parsing) only to decode data but we design our own encoding filters.

Encoding filter design

The idea behind filtering streams is quite simple and common so we use boost filtering iostreams which give us a very nice filtering input/output concept. Filtering stream consists of filters and devices. A device can be either a sink(input) a or a source(output). We need just buffer encoding so we use only sources (devices you can read from). We can connect arbitrary number of filters to source. When we read from a filtering input stream, the read call is delegated through the chain of filters to first device which returns raw character(s). Then every filter, in reverse order, encodes the character(s) and passes it to the next filter. Finally an encoded character(s) falls from the filtering input stream. Which filters to use is specified in stream dictionary.

Stream concept in pdfedit

This design is flexible, easily extensible as we only have to implement the new filter and connect it to the source when the filter name matches the filter name specified in the stream dictionary.

Generic observer mechanism described in the section called “Observers” is used in our cobject implementation. IProperty base class implements IObserverHandler interface and so all its descendants are responsible for notifying about changes (see the section called “Low level CObjects”.

All cobjects provides BasicChangeContext which contains previous value of changed one. Because complex types can add and remove its elements, special behaviour is specified. New value given as mandatory parameter of notify method may be CNull if element is deleted from comlex type. Old value in the context may also CNull if element was added. It is imposible to have both of them CNull.

CPdf class as PDF file maintainer uses observers for synchronization of structures which may be changed in two ways. This may occure because all attributes can be accesible through properties (cobjects) and also special objects provided by CPdf - CPage for page manipulation, COutline for outlines manipulation and so on. Special objects keeps logic of concrete entities and manipulates with cobject in that way. Property part is without any logic and enables making changes which are not covered by special objects. This advantage and extensibility is payed by additional synchronization.

	CPdf::insertPage
	Cpdf::removePage

Mode controller is class which provides us with property mode configuration. Each property keeps its mode. This is just information (kind of tag) which can be used by property user to determine how he should present it to application user. This is not any kind of restriction how property can be used.

The reason why to create such mode is that Pdf has many objects which are good for changes and some of them are just technical or more they contain meta data. Changes made to meta data or technical objects may lead to total corruption of whole document with very less effort (e. g. change of reference to page tree root leads to all pages lost). We didn't want to have what is possible and what not logic hardcoded in kernel. So we have defined mode for property and GUI which displays objects in property editor may look at property mode and decide whether it provides read only, visible or warning throwing access (GUI can also provide configuration how each mode should be treated).

ModeController class is responsible to provide mode to property according property name (id) and the type of parent type. All pdf properties are part of certain context (parent data). This context may be dictionary or array or object can be indirect and in so it is its own context. ModeController is based on RulesManager (see the section called “Rules manager”). As a rule data type, we use

struct ModeRule
{
	std::string type;
	std::string name;
	bool operator==(const ModeRule & rule)const
	{
		return type==rule.type && name==rule.name;
	}
};

structure. Type stands for context type (Type field value of parent data type or itself if object is indirect and so it has no parent) and name stands for name of property in context.

Property matcher implemented for ModeController defines 3 priorities of matching with following logic:

This enables to define rules with more or less general meaning and explicit setting for one property.

Inherited RulesManager enables configuration loading from stream by default. We have additionally implemented ModeConfigurationParser which is able to parse strings from configuration file. To be less dependant on particular configuration file format, we have built it upon StringConfigurationParser which simply reads key value pairs. from file and ModeConfigurationParser transforms this values to its internals data types for rule and target (ModeRule and Mode types). This is example of Adapter design pattern implementation.

We have written also example configuration file (this file is stored in data directory - by default /usr/share/pdfedit, but depends on prefix parameter during installation) with quite reasonablerestrictions. For more inforamtion about file format and mode types, please see project doxygen documentation for ModeController.

Mode controller and related classes.

Sometimes it is desireable to display progress of current operation to user. To accomplish this task, we have implemented ProgressObserver class which implements IObserver (see the section called “Observers”) and internally stores implementator of IProgressBar interface.

Responsibility of this class is to implement notify method so that current state of progress given as parameter is displayed by IProgressBar implementator. It has to recognize when progress starts (this is provided by internal flag), when it finishes (when scope's total count is reached).

Instance of this class can be registered on arbitrary descendant of ObserverHandler class which registers PdfWriterObserver ^[10] compatible observers and uses ScopeChangeContext with OperationStep and OperationScope template parameters.

Progress observer and related classes.

Content stream is a sequence of operations which alter graphical state. We need to obtain some information after each of these operations and xpdf code is no suitable for this sort of things. If appropriate functions are defined and the object is changed a bit, it could be used to display the content stream. It can be easily extended by adding appropriate operations when Xpdf code is very difficult to extend here because it is adjusted just for displaying purposes. Again it has almost zero fault tolerance which was improved by adding some constraints into xpdf code. Functions altering the graphical state were heavily inspired by xpdf code. Most of the code is just copied from xpdf.

Obtaining information after every operator

Each pdf operator has its function which alters the graphical state appropriately. The mechanism used by stateupdater is simple but very flexible. It gets a simple iterator which it uses to iterate through all operators. After performing a function assigned to the operator a functor with all needed information avaliable is called.

StateUpdater class

Setting operator bounding boxes

Kernel supplies a special functor to the stateupdater, which gets called after every operation with infromation about the rectangle the operator used. The functor sets this rectangle as the bounding box of the operator.

	virtual bool typeSafe(Object * obj1, Object * obj2);

For more information about responsibility and functionality separation see following figure.

3 layers diagram

As it was mantioned above, PDF format supports changes in document in so called incremental update (all changed objects are appened to document end and new cross reference section for changed objects). This means that each set of changes forms new revision. This brings little task to think about. What should be stored in one revision and which changes are not worth of new revision? User usually wants to save everything because of fear of data lost and doesn't thing about some revisions. If each save created it would lead to mess with horrible number of referencies without any meaning.

All previously mentioned functionality depends on incremental update mechanism. However pdf document may have format little bit different. Such documents are called Linearized and are designed for environment where it may be problem (e. g. time problem) to wait until whole document is read and so parsing from end of file can start (see Pdf specification Appendix F for more information).

Such documents have special requirements and they are not designed for making changes. 3rd layer handles this situation rather strictly and XRefWriter checks whether given file is linearized during initialization. Some of operations are not implmented in linearized document, such as revision handling and document saving may product not correct document (pdf viewers which strictly relies on linearized information may display different output).

Because many of documents (specialy from internet) are linearized, we have provided Delinearizator class placed in utils. It is able to get rid of linearized structures and create new pdf document which has same objects but normal structure. Usage of the class is very simple, see the following example:

        
        IPdfWriter * writer=new OldStylePdfWriter();
        Delinearizator *delinearizator=Delinearizator::getInstance(fileName.c_str(), writer);
        if(!delinearizator)
        {
            printf("\t%s is not suitable because it is not linearized.\n", 
                fileName.c_str());
            return;
        }
        string outputFile=fileName+"-delinearizator.pdf";
        printf("\tDelinearized output is in %s file\n", outputFile.c_str());
        delinearizator->delinearize(outputFile.c_str());
        delete delinearizator;

Menus are stored in gui section. Each menu, menu item, toolbar or toolbar item have a unique name (all these share common namespace) - also, menu items often can be used as toolbar items (as long as they have icon) and toolbar items can be used as menu items (as long as they have text).

items/name_of_list=list Caption of menu,menu item 1, menu item 2, ....

Caption of item

items_add/name_of_list=menu item 1, menu item 2, ....

items/name_of_item=item Caption of item,Command,Keyboard_shortcut,Icon

Icon

Caption of item

Command

Keyboard_shortcut

items/name_of_item=label Caption of label

PDF Editor support icon themes. Themes can be created by creating subdirectory in any directory from icon path and putting some custom icons there. Theme then can be activated by modifying the theme/current key in the icon section in configuration - set the key to name of the directory with theme. Or it can be simply specified in 'Icon theme' field in editor's configuration dialog.

Editor look first for icon from theme across all directories in icon path, by appending a theme name to the path - so for example if theme is hicolor and one icon directory is $HOME/.pdfedit/icon, editor looks in $HOME/.pdfedit/icon/hicolor for that directory. If a themed icon is not found, editor looks for a default icon in the icon directory itself (which also correspond to a default theme).

Static settings are stored in file pdfeditrc. This file is looked for at application start, first in data directory (which is defined in config.h in GUI source directory) and if not found there, next is searched the directory, where the editor executable is located. StaticSettings class handles reading of this file.

The format is very similar to the format of "ini file", but have been improved to allow indentation and comments, so the file is better readable and manageable.

The file is automatically assumed to be in utf-8 encoding. Any leading or trailing whitespaces on line are ignored, as are any whitespaces between key and equal sign, or between equal sign and value. Blank lines are ignored too. This allow to indent the file to be more human-readable. Basically, not counting comments and blank lines, the file may contain any number of lines with section identifiers and key-value pairs. Section identifier, in format [section path], sets prefix (i.e. the path in which the keys are stored in) for all following keys, until different section identifier is encountered. Key-value pair, in format key=value specifies one setting that maps value to specified key. Any number of whitespaces can be present directly behind and after the equality (=) character For example, value name = MyName in section specified by its heading [settings/part1] is then referenced in editor by its entire key name settings/part1/name and its value would be MyName

Static settings are stored in file pdfeditrc in subdirectory .pdfedit in user's home directory. Nota that most other settings (any custom scripts, icons, etc ...) are stored by default in that directory too. This file is read by QSettings class on application start and when last window is closed, the settings are writen back (if they were modified). Also, the file is explicitly written when user presses Ok or Apply button in Option dialog (to avoid losing settings if the program crashes or is killed) or if requested by script, for example by calling settings.flush().

The format is basically an "ini file". You can not insert any comments in the file or indent it, as it is not supported by the QSettings configuration file parser and the settings will be overwritten anyway when the file is updated.

Basically, the file may contain any number of sections, each section having one or more key-value pairs. Section identifier, in format [section path], sets prefix (i.e. the path in which the keys are stored in) for all keys in the section, while section named General have special meaning off being section with "empty prefix" (root section) After section identifier, the key-value pairs follow, one on line, in format key=value - they specify one setting that maps value to specified key. After last key-value pair in one section and the next section is one extra blank line. Putting extra blank lines in middle of the section (or anywhere else) is not allowed, as it will break the file format.

Class providing tree view of PDF objects. It does support multiple tabs, showing individual trees inside them.

Splitting the tree to multiple tabs partially solve the user disorientation problem, as all content streams are opened in tabs, thus their operator tree does not clutter the "main" tree view showing pages, annotations, outlines, etc ...

This window show list of tabs, with one "main" tab that contain the document as root element and zero or more "secondary" tabs, than show some elements from main tree more in detail. The main tree cannot be closed and is fixed to showing the PDF document as its root item. Secondary trees can be closed any time when the user think they are no longer needed.

Single tree in the multi tree window is managed by TreeWindow class

Class providing tree view of PDF objects, having one object at root and showing its children. It uses QListView for showing the tree and all items that are inside the QListView are derived from TreeItemAbstract class (which is derived from ordinary QListViewItem class) Also, the TreeWindow bring some limitation to the QListView in it, most notable, you can only put items that are derived from TreeItemAbstract class, not ordinary QListViewItem (if you bypass this limitation, you can expect strange behavior) and the listview must have at most one root item - this is required by GUI logic that tree correspond to something, either the document or some part of it (or the tree is empty). Also, it simplifies some things.

This class is inherited from basic QListViewItem, providing some extra functionality (getting QObject wrapper from the treeitem with a getObject() method, managing childs of the treeitem and some base methods to support automatically reloading only the necessary tree items when change is detected) All specific treeitems are derived from this class. The methods needed to fill tree with child items (if any) are purely virtual (abstract).