Using Remote Depots

When and when not to use remote depots

PROBLEM

When should I use a remote depot?

SOLUTION

Supporting Users at Remote Sites

New Perforce customers may assume that if their users are geographically distributed, they will need to set up separate Perforce installations connected by remote depot support. However, even distributed development organizations are likely to be more productive using a single Perforce installation.

An "installation" is a Perforce Server that manages any number of depots and client workspaces. The depots managed by a Perforce Server are local to the machine on which it runs. The files in those depots can be accessed by any user with a client workspace managed by that Perforce Server. Because Perforce is designed to cope with the latencies of long networks, it inherently supports users with client workspaces at remote sites. Thus a single Perforce installation can support a shared development project with contributors working from near and remote sites.

Remote depots, on the other hand, are designed to support shared code, not shared development. They enable independent organizations with their own Perforce servers to view files and integrate changes from depots in other installations. Remote depots are not the solution for load balancing or network access problems, although some insurmountable problems in these areas can be worked around using remote depots.

This tech note discusses issues concerning distributed development organizations, including:

• Supporting users at remote sites
• How remote depots work
• Limitations of remote depots
• Conclusions and recommendations

One Perforce server and no remote depots

A distributed organization sharing development of one or more software projects optimally has one Perforce Server, one or more Perforce depots, and a multitude of Perforce client workspaces. The geographic locations of the workspaces are irrelevant to the Perforce Server:

Number of Users Supported

Remoteness is not a factor in a Perforce Server's ability to support users. However, the number of users overall may be, and the network line speed and quality between the Perforce Server and the users' remote site certainly is.

At a remote site, the number of users who can work productively depends on the site's network connection. Using the data compression feature, available in Perforce 99.1 and later, it's estimated that a slow modem line would support 1-3 users, a 64KB line would support between 20-30 users, and a T1 line (1MB/sec) would support 100 or more users. A local ethernet (10MB/sec) would support whole companies. Of course, these estimates depend on the work users do. 40 users working with plain text source files could probably work happily on a 64KB line. But, especially without compression,  two users constantly updating large GIF images could saturate that line and make life miserable for everyone else.

If a single Perforce installation is supporting a large number of users, at sites near or far, you can reduce the load on your Perforce Server by using restrictive client views and access permissions so that each user is accessing only the depots and subdirectories he/she is working on.

Windows users at network-bound sites will be more efficient using p4, the Perforce Command-Line Client rather than P4Win, the Perforce Windows Client, because the current release of P4Win makes frequent behind-the-scenes data update requests from the Perforce Server.

Network Security

We recommend using the SSH (Secure Shell) package for punching through firewalls at both ends of your connections. You can set it up so that it is very secure, with public key encryption to verify the identities of the hosts at either end and then regular encryption to keep the data stream secure. What's more, SSH compresses while it encrypts, which should help with a slow modem line. See "Accessing Perforce Through a Firewall" for more information about SSH.

Multiple Perforce servers with remote depots

Without remote depot support, a client workspace can access only the depots in the installation it belongs to. The purpose of remote depot support is to allow clients to access depots in other installations:

The figures above show a Perforce installation that uses "mars:1666" as its server port address. Whether the "mars:1666" Perforce Server is running on a machine physically near you or not, if you set P4PORT to "mars:1666", it is your "local" Perforce Server when you use P4 or P4Win. The "mars:1666" Perforce Server manages the files in its depots, and keeps a database of changelists, integration records, client workspaces, and other metadata. All the metadata in that Perforce Server is available to you, and you may submit changes to the files in that Perforce Server's depots, as long as you have the right access permissions. The depots managed by "mars:1666" are your "local" depots.

Say you wanted to access files in a depot managed by the "venus:1666" Perforce Server. You could set P4PORT to "venus:1666", and that would make "venus:1666" your local Perforce Server. Your P4 and P4Win commands would now communicate with that Perforce Server. But because the "venus:1666" Perforce Server doesn't know about the client workspace you were using when "mars:1666" was your local Perforce Server, you'd have to set up a new client workspace. In fact, the "venus:1666" Perforce Server knows nothing about your relationship with the "mars:1666" Perforce Server: it doesn't know what access permissions you have, what your client view is, what files you have opened in your workspace, etc. So in the case where you want to compare or merge files from a "venus:1666" depot with files in a "mars:1666" workspace, connecting to "venus:1666" as your local Perforce Server doesn't help you.

The Perforce remote depot support allows you to access files from the "venus:1666" depot while you are connected to "mars:1666" as your local Perforce Server. For instance, you could define a remote depot called "//venus" with a mapping to the "venus:1666"  depot path "//depot/...". Once you've done this, you can use p4 commands to refer to files in the //venus path. For example:

p4 diff2 //venus/elm2.1/... //depot/elm2.1/...

Will diff all the files in the on the "//depot2/elm2.1" path on "venus:1666"  with their counterparts in the "//depot/elm2.1" path on "mars:1666".

You can also sync your workspace with files from "//venus", branch files from "//venus" into a local depot, and integrate changes from "//venus" into a local depot.

When you access files in the "//venus" path, p4 is not communicating directly with the "venus:1666" Perforce Server. Instead, your p4 (or P4Win) program communicates with the "mars:1666" Perforce Server, which in turn communicates with the "venus:1666" Perforce Server.

Limitations of Remote Depots

Access is read only. Remote depot access is restricted to read-only operations. You cannot submit files into a remote depot. You cannot edit files from the remote depot. You can, however, create a branch in a local depot from files in a remote depot, and then integrate changes from the remote depot into the local branch. This is advantageous in situations where you want to control when and how code is changed, such as when working with a third party developer.

Server version determines remote depot behavior: Beginning with server version 99.2, remote depots will interoperate between UNIX and Windows. Beginning with version 98.2, remote depots will interoperate across (98.2 and higher) releases. In 98.1 and before, remote depots will only operate with another Perforce Server of the same release.

Licensing requirements: You do not need additional licenses to access remote depots.

No metadata access: A Perforce Server's metadata (i.e., information about client workspaces, changelists, labels, etc.) can't be accessed using remote depots. Commands like p4 client always return specifications of entities defined in the local Perforce Server's metadata. This can also be handy when sharing code with a third party, as it prevents them from seeing internal company information.

Referencing file revisions: Files in a remote depot may be specified by revision number or changelist number, but if specified by a changelist number, the remote Perforce Server's changelist number is used, not the local Perforce Server's. Files in a remote depot may be accessed by label name or client name, but the local Perforce Server's label and client specifications will be used in these cases.

One-sided integration records: Because each Perforce Server sees only its own metadata, its knowledge of integrations between depots in separate installations is severely limited. For example:

• The "m_foo.c#15" file somewhere in a "mars:1666" depot is branched into "v_foo.c#1" somewhere in a "venus:1666" depot.
  
  
• A local user modifies the file on "venus:1666" so that it's now "v_foo.c#2".
  
  
• A user on "mars:1666" integrates from "v_foo.c" to "m_foo.c". Now "m_foo.c#16" contains the "v_foo.c#2" delta.
  
  
• Another user on "venus:1666" integrates from "m_foo.c" to "v_foo.c". But, since the "venus:1666" Perforce Server can't see the "mars:1666" metadata, it doesn't know that the "m_foo.c#16" delta came from "v_foo.c#2". All it knows is that the last integration was done from "m_foo.c#15". So it schedules an unnecessary integration from "m_foo.c#16" to "v_foo.c".

Performance: Remote access can seriously degrade local performance. If a remote Perforce Server is being accessed and the connection is very slow, local users will hang while the connection is made and the data brought back. It is best to protect the remote depot from the general public with the p4 protect command, and limit its access to only specific users who understand that slowdowns can occur.

Conclusions and Recommendations

Whenever possible, Use a single Perforce installation for distributed development projects. Perforce is designed to support remote users. If users at geographically separated locations are in fact working on the same development project, all their code should be in depots managed by one Perforce Server. Trying to partition joint development projects into separate Perforce installations does not improve throughput, and usually only complicates administration. And if your network is so slow that Perforce, a light-footed, passive client-server application, can't solve your joint development problems, no remote or multi-site implementation will improve your productivity as much as upgrading your network will.

Use remote depots to import work in progress from other organizations. There are cases where, even though Group B does not contribute to the development of Group A's project, it needs access to Group A's latest code. Again, this is what remote depot support was designed for: to share code, not development, across organizations.

Restrict client views and access permissions. If your installation has a large number of users and/or a large number of depot files, you can optimize Perforce Server response time by limiting each connecting user's view of the files. If you use remote depots, use restrictive depot views as well as restrictive client views and access permissions to reduce the effect of remote depot queries on local users. Users can tailor their own client specifications to restrict their views to the depot files they are working on. Perforce superusers can restrict user access to depot files using protections.