Maximizing Perforce Performance

Views, Protections, Maxresults, Maxscanrows, and Maxlocktime

TASK

How can I maximize performance using views, protections, maxresults, maxscanrows, and maxlocktime?

SOLUTION

The Perforce Server's performance depends on the number of files you try to manipulate in a single command invocation, not on the size of the depot. For example, syncing a client view of 30 files from a 3,000,000-file depot is about as fast as syncing a client view of 30 files from a 30-file depot.

The number of files a single command affects is determined by:

• p4 command line arguments (or selected folders, in the case of P4V or P4Win commands). Without arguments, a command operates on all files mapped by the client view.
• Client views, branch views, label views, and protections. With unrestricted views and unlimited protections, a command operates on all files in the depot.

At sites where depots are very large, unrestricted views and unqualified commands make a Perforce Server work much harder than it needs to. When the Perforce Server answers a request, it locks the database for the duration of the computation phase. For normal operations, this is a successful strategy, as the Perforce Server can process requests quickly enough to avoid requests piling up. But abnormally large requests can take seconds, sometimes even minutes. If frustrated users hit CTRL-C and retry, the problem gets worse -- the Perforce Server starts using lots of memory and responds even more slowly.

Perforce users and administrators can prevent Perforce Server thrashing by:

• Using "tight" views
• Assigning protections
• Setting maxresults to limit the amount of data affected
• Setting maxscanrows to limit the number of revisions scanned
• Setting maxlocktime to limit the amount of time spent during data scans

These options are described in more detail in the sections below.

Using "tight" views

Define client views (and branch and label views) so users access only the files they require. For example, the following "tight" client view is restricted to specific depot areas:

Example

//depot/main/svr/devA/...       //ws/main/svr/devA/...
//depot/main/dvr/lport/...      //ws/main/dvr/lport/...
//depot/rel2.0/svr/devA/bin/... //ws/rel2.0/svr/devA/bin/...
//depot/qa/s6test/dvr/...       //ws/qa/s6test/dvr/...

By contrast, the following unrestricted view is easier to set up but invites trouble when depots are very large:

//depot/...   //ws/...

Client views, branch views, and label views are defined using the p4 client, p4 branch, and p4 label commands, respectively, by the users who created them.

Assigning protections

Protections are a type of view. Protections are set using p4 protect command, and they control which depot files can be affected by the commands that users run. Unlike views, however, protections can be set only by superusers. Protections also control read and write permission to depot files, but permission levels themselves have no impact on Perforce Server performance.

Protections can be assigned either to users or to groups.

Example

write user  sam        * //depot/admin/...
write group rocketdev  * //depot/rocket/main/...
write group rocketrel2 * //depot/rocket/rel2.0/...

Groups are created by superusers with the p4 group command. Groups make it easier to assign protections and enable you to set three performance-tuning options -- maxresults, maxscanrows, and maxlocktime -- described next.

Setting maxresults to limit the amount of data affected

Each group has a maxresults value associated with it. The default value is "unlimited", but a superuser can use p4 group to limit maxresults for a particular group. Users in the group cannot run any commands that affect more database rows than the maxresults limit. For most commands, the number of database rows affected is roughly equal to the number of files affected.

Example

Group: rocketdev
Maxresults: 10000
Users:
       bill
       ruth
       sandy

Ruth has an unrestricted client view. When she tries:

p4 sync 

her p4 sync command is rejected if the depot contains more than 10,000 files, and she will see an error message at her client such as:

Request too large (over 10000); see 'p4 help maxresults'.

She can get around this by syncing smaller sets of files at a time, for example:

p4 sync //depot/projA/...
p4 sync //depot/projB/...

She will get her files, without tying up the server to process a single, extremely large command.

Setting maxscanrows to limit the number of revisions scanned

Each group also has a maxscanrows value associated with it. The default value is "unlimited", but a superuser can use p4 group to limit maxscanrows for a particular group. Users in the group cannot run any commands that scan more revisions than the maxscanrows limit. This option is particularly useful for limiting wildcarded commands that would otherwise scan every revision, such as:

p4 files //...foo...

Example

Group: rocketdev
MaxScanRows: 50000
Users:
       bill
       ruth
       sandy

If there are more than 50,000 revisions in all depots and Bill issues the following command:

p4 files //...foo...

his command is rejected and the following message is seen at the client:

Too many rows scanned (over 50000); see 'p4 help maxscanrows'.

But if Bill narrows the scope of his command so it scans fewer than 50,000 revisions, it succeeds:

p4 files //depot/main/...foo...
//depot/main/foo#4986 - edit change 4994 (text)

With the scope narrowed, the impact on the server is far less.

The maxscanrows limit does not affect the following commands:

• integrate
• labelsync
• obliterate
• resolve
• revert
• submit

Setting maxlocktime to limit the amount of time spent during data scans

Each group also has a maxlocktime value associated with it. The default value is "unlimited", but a superuser can use p4 group to limit maxlocktime for a particular group. The maxlocktime limit is entered in milliseconds. Maxlocktime operates by starting a counter when the first table is (read) locked, and includes time waiting for any subsequent table readlocks. Because the number of users and commands run at any one time varies, this option will vary in behavior due the load on the server. If the time the readlocks are held/waited for a command exceeds the group maxlocktime, the command will fail.

Example

Group: rocketdev
MaxLockTime: 30000
Users:
       bill
       ruth
       sandy

Starting with the first readlock taken for the command, if the locks are held (including waiting) for a time exceeding maxlocktime, the command will fail.

p4 opened ...

if the readlocks taken exceed 30000 (milliseconds = 30 seconds)

Operation took too long (over 30.00 seconds); see 'p4 help maxlocktime'.

The server log will include details about the command within the Trace output:

--- killed by MaxLockTime

For more information:

• For information about how maxresults, maxscanrows, and maxlocktime are determined for users who belong to more than one group, see Tech Note 49.
• To display an informative comparison of commands and the amount of data they affect, use p4 help maxresults.
• For a discussion of performance and tuning issues, and protections and access permissions, see the Perforce System Administrator's Guide.
• Usage notes and syntax rules for the p4 commands mentioned above can be found in the Perforce Command Reference.