Using p4 -G

Marshalled Python output

SUMMARY

The global -G flag generates command output as marshalled Python dictionaries. This article offers basic guidance for using the global -G flag for scripting Perforce.

DETAILS

As noted in p4 help usage:

The -G flag causes all output (and batch input for form commands
with -i) to be formatted as marshalled Python dictionary objects.

The output dictionaries (dict) contain fields corresponding to the output of the p4 command and any API information.

The output dictionary's fields differ from command to command, but most have a "code" field that contains one of three things: "stat", "error" or "info". "stat" means "status", and is the default result. "error" means an error has occurred - the full error message is contained in the "data" field. "info" means that there was some feedback from the command - it too is placed in the "data" field.

The fields of the dict, in most cases, correspond to the resulting output field. For example, "Description" matches the Description of a client spec returned from p4 clients. "fromFile" matches the source filepath in an integration command.

It can be useful to dump the contents of the dict to see what fields are available to you. Most of the fields are human-readable and clearly indicate what they correspond to, however, others are not so clear, and you will need to reference the Perforce API.

Examples and Troubleshooting

When you use p4 -G, you need to be aware of three common pitfalls:

• Failing to loop on the input stream (required in order to read multiple records)
• Not opening the input stream in binary mode on Windows (when using popen)
  
• Not specifying the marshal format version number on output

Example

This example demonstrates running an interactively specified Perforce command, reading the output (a stream of marshalled dictionary objects) in a loop, and then printing the results. This example also shows how to replace the deprecated os.popen call with the new subprocess.Popen introduced in Python 2.6.

import os, marshal
import sys

cmd = raw_input( "Enter a Perforce command (omitting 'p4 -G'): " )

#
# basic dictionary read loop using -G flag
#
list = []
if sys.version_info[1] < 6:
  pipe = os.popen( 'p4 -G ' + cmd, 'r' ) # 'rb' on Windows for binary read
else: # os.popen is deprecated in Python 2.6+
  from subprocess import Popen, PIPE
  pipe = Popen( 'p4 -G ' + cmd, stdout=PIPE).stdout
try:
    while 1:
        record = marshal.load( pipe )
        list.append( record )
except EOFError:
    pass
pipe.close()

# print list of dictionary records
c = 0 
for dict in list:
    c = c + 1
    print "\n--%d--" % c
    for key in dict.keys():
        print "%s: %s" % ( key, dict[key] )

Example

This example demonstrates reading p4 -G output from stdin and prints the resulting sequence of keys and values. This script produces the same output for a given command as the previous example.

#!/usr/bin/env python

import sys, marshal

try:
    num=0
    while 1:
        num=num+1
        print '\n--%d--' % num
        dict =  marshal.load(sys.stdin)
        for key in dict.keys(): print "%s: %s" % (key,dict[key])

except EOFError: pass 

Example

The following example class demonstrates using the os.popen2() method to create a pair of pipes in binary mode allowing you to read from and write to the p4 -G process. As in the first example, the following code uses the now deprecated popen2() for Python releases before 2.6 and subprocess.Popen() for Python 2.6 +.

  #!/usr/bin/python
  import os, marshal, sys

  class P4:
      def run( self, cmd, args=[], input=0 ):
          c = "p4 -G " + cmd + " " + " ".join( args )
	  print( c + "\n" )
	  if sys.version_info[1] < 6:
            (pi, po) = os.popen2( c , "b" )
          else:
            from subprocess import Popen, PIPE
            p = Popen( c, stdin=PIPE, stdout=PIPE )
            (pi, po) = (p.stdin, p.stdout)
	  if input:
              marshal.dump( input, pi, 0 )
              pi.close()
          r = []
        try:
            while 1:
        	x = marshal.load( po )
        	r = r + [ x ]

        except EOFError: pass
        return r

if __name__=="__main__":
        p4class=P4()
        r = p4class.run("clients")
        for record in r:
                print "-------"
                for item in record.keys():
                        print "%s: %s" % (item, record[item])

Notes
When using "marshal.dump()" in Python 2.4 or later, you must specify version "0" of the dump format or P4 cannot understand the data that is being sent.

An alternative to p4 -G is to use P4Python, which is an integration of the Perforce commands into Python as a module. You can find more details here: P4Python