The initial implementation of a federated CORK server will focus on
supporting caching and firewall traversal. As such, it will ignore some potential security issues and not attempt to aggressively optimize network traffic.
In the initial prototype:
- Authentication will be delegated to the master server (MS). Ideally this would not require a separate connection from the caching server (CS) to the MS for each user connected to the CS. ClientIDs are currently assigned on login, so the MS can continue to be the source of clientIDs for clients connected to the CS.
- A cache of objects will be shared by all clients connected to the CS.
- If a request appears for an object that is not in the CS cache, the request will be delegated to the MS using the client's credentials. The object will be placed in the CS cache and then sent to the requesting client.
- If a request appears for an object that is in the CS cache, the object permissions will be checked to ensure that the client is actually allowed to retrieve the object.
- Objects will be cleared from the CS cache only when there are no connected clients using the object. A later refinement of the CS design could include temporary persistant storage of objects from the cache in order to reduce the memory footprint of the CS.
- New object additions will first be applied in the CS, and then propagated to the MS. This means that the CS (and not the MS) will be responsible for generating an ObjectID for the new object.
- Delete requests will first be processed in the CS cache (following a permissions check) and then sent to the MS.
- When a Change is recieved by the CS from the MS, it will first be applied to the cached version of the object, and then propagated to any connected clients that are using the object.
- When a Change is broadcast by a client connected to the CS, the CS will not immediately apply the change to the cached copy of the object. Instead it will
- Save a reference to the change, along with information describing whether or not the change should be broadcast back to the sender
- Re-broadcast the change to the server, telling the server to the send the change back to the sender.
- Apply the change to the cached copy of the object when it is sent back to CS from the MS.
- Broadcast to the change to any connected clients using the object, including the originating client if send-to-all was originally specified.
- Permissions changes originated by a client connected to the CS will be handled in the same way as object changes. This will ensure that an invalid permissions change will be stopped at the MS and not applied to objects cached in the CS.
- The CS will notify the MS when a connected client logs out.
Change to the existing CORK infrastructure will include:
- Support for multiplexed connections will need to be added. These will need the ability to have multiple authenticated users associated with them, as well as unique client ids for each user. (So that
isClientAlive calls will work.)
- Incoming Change objects are currently associated with the user who has authenticated the connnection. The message used to broadcast changes will need to be extended to allow the originating user to be specified.
- A new message will be needed for adding an object to the MS that includes an ObjectID generated by the CS.
- A new message will need to be added that allows the CS to notify the MS that a connected client has disconnected.
Multiplexed connections
From the perspective of the central server, per-CS-connected-user records are needed for:
- Principals (so that incoming changes can be appropriately attributed)
- client ids, so that isClientAlive(...) calls work
Ideally, there would only be one of these per CS, shared by all users connected to the CS:
- ServerSkel, so that only a single socket and thread would be used
- ClientQueue, so that only a single copy of outgoing objects would have to be managed
The ServerImpl class assumes that each connection is associated with a single user. A proposed solution is:
- MultiplexServerSkel (MSS) listens on a separate port. (Need to see if this can be avoided, for firewall configuation simplicity.)
- When a request for authentication arrives from the CS, the MSS passes the request to the Server, registering a proxy client object. The server will assign a client id to the proxy client and give it a queue for outgoing changes.
- All other incoming messages from the CS will be routed to the server by the MSS
- Outgoing broadcastChange messages from the server will be intercepted by the MSS and "pruned" so that only a single instance of the Change will be sent to the CS in the case where multiple users on the CS may be using the object.
- This is a problem, since it may not be practical to track the changes since queue sizes may vary considerably
This approach limits the changes that will have to be made to the server and minimizes network traffic. The drawbacks are:
- The server is forced to do a small amount of extra effort to broadcast changes that may be thrown away by the MSS.
- Filtering outgoing changes will be problematic
- Listening on a separate port will definitely be a practical issue (given firewalls, for example)
An alternate approach would be:
- Allow GenericServerSkel to support multiple authenticated users. This would mean:
- Storing a list of ReplicatedObjectUsers instead of a single user object
- Modifying
ServerImpl.addClient(...) to support multiple client ids for a single queue:
- On initial connection from a CS, the GenericServerSkel could request a special client id that was not associated with a user. A ClientQueue would be associated and put in the list of clients. The queue will be associated with any user clientIDs created from subsequent logins from clients attached to the CS.
- ClientQueue would have to associate users with object-ids-in-use in order to handle releaseObject(...) calls and client logouts. This means that it must be possible to quickly adjust the users associated with a given object id (to handle addID and removeID), and also to clear all associations for a given user.
- A map of users to lists of ids would handle the logout case
- A map of ids to lists of users would handle the addID, removeID case
- Currently there are cases where the server only knows the clientID for a user, so it will be cheaper to represent users by client id than by name/principal/etc...
- Allow incoming SocketMessages to (optionally) specify the originating user. If the specified user has been authenticated on the connection, GenericServerSkel will use it to execute the message. To prevent a possible performance hit, this should probably be implemented as a wrapper around SocketMessage. (Only messages from a CS would use the wrapper.)
CachingServer
The Caching server should be able to implement the Server interface, and use existing classes like SocketServer and SocketServerSkel.
- On startup, a connection to the MS will be made.
- If the connection to the MS is broken, the CS should drop all connections to clients and exit. Typically, the CS would be running in a script that would sit in a loop, sleeping for a few seconds after server termination, then checking for updated jars, then restarting.
|