Project Idea | Collaborative Editor Framework in Real Time

NOTE: Most of the contents are taken from Project Report.So you can also download it from drop box links are given down in (Following additional topics may also be provided) Block . Simply you can read it out .

Collaborative Editor Framework in Real Time

Introduction:

A collaborative editor is a form of collaborative software application that allows several people to edit a document or a program collaboratively over a computer network. In real time collaborative editing users can edit the same file simultaneously, whereas in Non-real time collaborative editing, the users do not edit the same file at same time.Since there is very small number of open source solutions to achieve collaborative editing in any web application, we intend to develop a framework in JavaScript through which anyone can achieve this functionality easily.In order to solve the problem of concurrency in such a collaborative editing framework, we will be using operational transformation.

Also in order to improve upon the traditional approach, we will consider using a hierarchical representation of text rather than the linear one.Real-time operation is an important aspect to be considered in the design of collaborative editors as users should be able to see the effects of their own actions immediately and those of other users as soon as possible. We will achieve this through web sockets

The problem of concurrency in collaborative editing is a well researched problem. People have suggested turn taking algorithm (Greenberg, 1991) allowing only one active participant at a time. Such an algorithm lacks concurrency and locks the document.

Other approaches like locking (Greenberg and Marwood, 1994) guarantees that users access objects in the shared workspace one at a time. Concurrent editing is allowed only if users are locking and editing different objects.
The operational transformation approach has been identified as an appropriate approach for maintaining consistency of the copies of the shared document in real-time collaborative editing systems

Google Wave, a real-time collaboration tool which has been in open beta for the last several months.

The treeOPT algorithm (Ignat and Norrie, 2003) improves upon operational transformation by considering hierarchical representation of the shared document in their research paper.[2] The approach is claimed to be increasing the efficiency of operational transformation compared to traditional approaches.

Conceptual framework:

Methodology
In order to ensure high responsiveness we will be using a replicated architecture where users will work on their own copy of the shared document. Only changes done are transmitted to other users.
Here is the control flow of such architecture:

• Each collaborator will hold a copy of the shared document along with the server.
• Whenever any of the shared copies will be edited, operations are calculated and transmitted to the server. To ensure real time transmission, we will leverage the power of web sockets.
• In case different users are updating different part of the same document, server applies the changes to its copy and forwards the changes to other collaborators.
• In case different users are trying to update the same part of document (for e.g. trying to add string to same location), i.e., concurrency occurs, server will need to resolve the race condition and one of the collaborators will be able to commit its changes.
• Now the actual problem starts. The server’s copy of the document and copy of the other clients have diverged.

The above problem of server’s copy and client’s copy being diverged can be represented through this image. Suppose server applied operation b on the document and client applied operation a on the document. This has caused both the copies to “diverge” from each other. In order to resolve the problem, or to “converge” both the copies to same version, we will use operational transformation.

Operational transformation adjusts operation a with respect to b and operation b with respect to a. The final results must be such that the adjusted operations cause the diverged copies to converge. Mathematically this can be written as:

OT(a, b) = (a', b') such that:
apply(apply(str, a), b') = apply(apply(str, b) 

In other words transformation function takes two operations, one server and one client and produces a pair of operations. These operations can be applied to their counterpart’s end state to produce exactly the same state when complete . As the fig 2 shows, after applying operation b’ to client’s copy and operation a’ to server’s copy they are now in same state.
Using operational transformation on a linear representation of text is not very efficient. In order to improve performance we will use the treeOPT algorithm’s approach. We will represent the text in hierarchical order

Using operational transformation on a linear representation of text is not very efficient. In order to improve performance we will use the treeOPT algorithm’s approach. We will represent the text in hierarchical order.

In such hierarchical representation, each node holds its own history. Operational transformation can now be applied on respective node rather than on the entire document. This reduces the overhead of going through history of entire text, as now we just have to go through history of only respective node.

Data structures and Algorithms

Tree operational Transformation Algorithm

Definitions

Node : A node N is a structure of the form N =, where   
Level: is a granularity level, level?{0, 1, 2, 3, 4},
corresponding to the element type represented by node 
(i.e. document, paragraph, sentence, word or character)   
Children: is an ordered list of nodes 
{child1, ..., childn},   level(childi)=level+1, for all i?{1, ..., n}  
Length : is the length of the node,
 
Length = {1, if level = 4 
         ? length(childi), otherwise 
         }
 
History: is an ordered list of already executed operations on children nodes  
Content: is the content of the node, defined only for leaf nodes  

Content = { undefined,  if level < 4 
           aCharacter, if level = 4 
           } 

Composite Operation:  A composite operation 
is a structure of the form cOp=, where: - 
Level: is a granularity level, level?{1, 2, 3, 4}  
Type:  is the type of the operation, type?{Insertion, Deletion}  
Position:  is a vector of positions   
position[i]= position for the ith    granularity level,  i?{1, ..., level}
- content is a node representing the content of the operation   
StateVector: is the state vector of the generating site 
Initiator: is the initiator site identifier.

Given a new causally ready composite operation, cOp, the root node of the hierarchical representation of the local copy of the document, rootNode, and the number of levels in the hierarchical structure of the document, noLevels, the execution form of cOp is returned.

treeOPT(rootNode, cOp, noLevels) { 
currentNode = rootNode;  
for (l = 1; l <= noLevels; l++)   
onew = Composite2Simple(cOp, l); 
eonew = Transform(onew, history(currentNode)); 
position(cOp)[l] = position(eonew);  
if (level(cOp) = l)      
return cOp;   
           currentNode = childi(currentNode), where i=position(eonew);  
}  
 

Client – server communication protocol

In order to ensure fast transfer of operation from one client to another we used websockets between client and servers.
In order to avoid polling the database on repeated intervals, we used mongoDB which allows us to subscribe to live feed of database tables.
Client is made using java and server is based on nodejs . Server works on

 
$ node www { address: '0.0.0.0',  family :  'IPv4',  port: 3000  } 

Client request server to share file, now server will share the file though FileID present in MongoDB . It then add user to the requested client’s session using session manager and also can remove user from it.
Whenever a change is made by client, operations are sent to history collection and client waits for server’s acknowledgement.
If there are further changes on client’s side, they are stored in buffer until the server’s acknowledgement comes. All the changes in buffer are sent once the server acknowledges.

Tools Used:

• Node.js: It is a JavaScript runtime built on chrome’s V8 JavaScript engine. Node.js uses an event-driven, non-blocking I/O model that makes it efficient.
• MongoDB: is a free and open-source cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with schemas.
• JavaFx: used for making client GUI.
• Robomongo: is a shell-centric cross-platform MongoDB management tool. Unlike most other MongoDB admin UI tools, Robomongo embeds the actual mongo shell in a tabbed interface with access to a shell command line as well as GUI interaction.

Application:
A real time collaborative integrated development environment can provide developers with the facility to collaborate over software projects over a network even when developers are thousand of miles away. Real-time Collaborative IDE provide developers with the ability to collaboratively write code, build and test it as well as share their projects with other developers. Chatting with other fellow developers over a project is also possible. Besides several other useful features of a complete IDE including saving snapshots, project management are also provided to ease the entire project development process.

Following additional topics may also be provided:
1) Github –https://github.com/agnu13/Collaborative-Editor-client
All the description for running project is given in github repository

2) Project Report and Presentation – Project Documents

3)Base Paper – TreeOPT algorithm: Customizable collaborative editor relying on treeOPT algorithm(Ignat and Norrie, 2003)

References

• Images: [1]. Divergence of server and client copies from online source: http://www.codecommit.com/blog/java/understanding-and-applyingoperational-transformation

Citations:

• [1].Introduction to collaborative editors. From Wikipedia: https://en.wikipedia.org/wiki/Collaborative_real-time_editor
• [2].In favour of tree representation, excerpt taken from research paper on treeOPT algorithm: Customizable collaborative editor relying on treeOPT algorithm(Ignat and Norrie, 2003)
• [3].Explanation of basic OT, excerpt taken from online source: http://www.codecommit.com/blog/java/understanding-and-applyingoperational-transformation