Previous topic

The nova.testing.runner Module

Next topic

The nova.version Module

This Page

Psst... hey. You're reading the latest content, but it might be out of sync with code. You can read Nova 2011.2 docs or all OpenStack docs too.

The nova.utils Module

Utilities and helper functions.

class GreenLockFile(path, threaded=True)

Bases: lockfile.LinkFileLock

Implementation of lockfile that allows for a lock per greenthread.

Simply implements lockfile:LockBase init with an addiontall suffix on the unique name of the greenthread identifier

class LazyPluggable(pivot, **backends)

Bases: object

A pluggable backend loaded lazily based on some value.

class LoopingCall(f=None, *args, **kw)

Bases: object

start(interval, now=True)
stop()
wait()
exception LoopingCallDone(retvalue=True)

Bases: exceptions.Exception

Exception to break out and stop a LoopingCall.

The poll-function passed to LoopingCall can raise this exception to break out of the loop normally. This is somewhat analogous to StopIteration.

An optional return-value can be included as the argument to the exception; this return-value will be returned by LoopingCall.wait()

class ProtectedExpatParser(forbid_dtd=True, forbid_entities=True, *args, **kwargs)

Bases: xml.sax.expatreader.ExpatParser

An expat parser which disables DTD’s and entities by default.

entity_decl(entityName, is_parameter_entity, value, base, systemId, publicId, notationName)
reset()
start_doctype_decl(name, sysid, pubid, has_internal_subset)
unparsed_entity_decl(name, base, sysid, pubid, notation_name)
class UndoManager

Bases: object

Provides a mechanism to facilitate rolling back a series of actions when an exception is raised.

rollback_and_reraise(msg=None)

Rollback a series of actions then re-raise the exception.

Note

(sirp) This should only be called within an exception handler.

undo_with(undo_func)
advance_time_delta(timedelta)

Advance overriden time using a datetime.timedelta.

advance_time_seconds(seconds)

Advance overriden time by seconds.

bool_from_str(val)

Convert a string representation of a bool into a bool value

check_isinstance(obj, cls)

Checks that obj is of type cls, and lets PyLint infer types.

cleanup_file_locks()

clean up stale locks left behind by process failures

The lockfile module, used by @synchronized, can leave stale lockfiles behind after process failure. These locks can cause process hangs at startup, when a process deadlocks on a lock which will never be unlocked.

Intended to be called at service startup.

clear_time_override()

Remove the overridden time.

convert_to_list_dict(lst, label)

Convert a value or list into a list of dicts

current_audit_period(unit=None)

This method gives you the most recently completed audit period.

arguments:
units: string, one of ‘hour’, ‘day’, ‘month’, ‘year’
Periods normally begin at the beginning (UTC) of the period unit (So a ‘day’ period begins at midnight UTC, a ‘month’ unit on the 1st, a ‘year’ on Jan, 1) unit string may be appended with an optional offset like so: 'day@18‘ This will begin the period at 18:00 UTC. 'month@15‘ starts a monthly period on the 15th, and year@3 begins a yearly one on March 1st.
returns: 2 tuple of datetimes (begin, end)
The begin timestamp of this audit period is the same as the end of the previous.
debug(arg)
default_flagfile(filename='nova.conf', args=None)
delete_if_exists(pathname)

delete a file, but ignore file not found error

deprecated(message='')

Marks a function, class, or method as being deprecated. For functions and methods, emits a warning each time the function or method is called. For classes, generates a new subclass which will emit a warning each time the class is instantiated, or each time any class or static method is called.

If a message is passed to the decorator, that message will be appended to the emitted warning. This may be used to suggest an alternate way of achieving the desired effect, or to explain why the function, class, or method is deprecated.

dumps(value)
execute(*cmd, **kwargs)

Helper method to execute command with optional retry.

If you add a run_as_root=True command, don’t forget to add the corresponding filter to nova.rootwrap !

Parameters:
  • cmd – Passed to subprocess.Popen.
  • process_input – Send to opened process.
  • check_exit_code – Single bool, int, or list of allowed exit codes. Defaults to [0]. Raise exception.ProcessExecutionError unless program exits with one of these code.
  • delay_on_retry – True | False. Defaults to True. If set to True, wait a short amount of time before retrying.
  • attempts – How many times to retry cmd.
  • run_as_root – True | False. Defaults to False. If set to True, the command is prefixed by the command specified in the root_helper FLAG.
Raises:
  • exception.Error – on receiving unknown arguments
  • exception.ProcessExecutionError
Returns:

a tuple, (stdout, stderr) from the spawned process, or None if the command fails.

fetchfile(url, target)
find_config(config_path)

Find a configuration file using the given hint.

Parameters:config_path – Full or relative path to the config.
Returns:Full path of the config, if it exists.
Raises :nova.exception.ConfigNotFound
flatten_dict(dict_, flattened=None)

Recursively flatten a nested dictionary.

gen_uuid()
generate_glance_url()

Generate the URL to glance.

generate_mac_address()

Generate an Ethernet MAC address.

generate_password(length=20, symbolgroups=('23456789', 'ABCDEFGHJKLMNPQRSTUVWXYZ', 'abcdefghijkmnopqrstuvwxyz'))

Generate a random password from the supplied symbol groups.

At least one symbol from each group will be included. Unpredictable results if length is less than the number of symbol groups.

Believed to be reasonably secure (with a reasonable password length!)

generate_uid(topic, size=8)
get_from_path(items, path)

Returns a list of items matching the specified path.

Takes an XPath-like expression e.g. prop1/prop2/prop3, and for each item in items, looks up items[prop1][prop2][prop3]. Like XPath, if any of the intermediate results are lists it will treat each list item individually. A ‘None’ in items or any child expressions will be ignored, this function will not throw because of None (anywhere) in items. The returned list will contain no None values.

get_my_linklocal(interface)
hash_file(file_like_object)

Generate a hash for the contents of a file.

import_class(import_str)

Returns a class from a string including module and class.

import_object(import_str)

Returns an object including a module or module and class.

is_older_than(before, seconds)

Return True if before is older than seconds.

is_uuid_like(val)

For our purposes, a UUID is a string in canonical form:

aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa

is_valid_cidr(address)

Check if the provided ipv4 or ipv6 address is a valid CIDR address or not

is_valid_ipv4(address)

valid the address strictly as per format xxx.xxx.xxx.xxx. where xxx is a value between 0 and 255.

isotime(at=None)

Stringify time in ISO 8601 format

last_octet(address)
loads(s)
logging_error(*args, **kwds)

Catches exception, write message to the log, re-raise. This is a common refinement of save_and_reraise that writes a specific message to the log.

make_dev_path(dev, partition=None, base='/dev')

Return a path to a particular device.

>>> make_dev_path('xvdc')
/dev/xvdc
>>> make_dev_path('xvdc', 1)
/dev/xvdc1
map_dict_keys(dict_, key_map)

Return a dict in which the dictionaries keys are mapped to new keys.

monkey_patch()

If the Flags.monkey_patch set as True, this function patches a decorator for all functions in specified modules. You can set decorators for each modules using FLAGS.monkey_patch_modules. The format is “Module path:Decorator function”. Example: ‘nova.api.ec2.cloud:nova.notifier.api.notify_decorator’

Parameters of the decorator is as follows. (See nova.notifier.api.notify_decorator)

name - name of the function function - object of the function

normalize_time(timestamp)

Normalize time in arbitrary timezone to UTC

novadir()
parse_isotime(timestr)

Turn an iso formatted time back into a datetime.

parse_mailmap(mailmap='.mailmap')
parse_server_string(server_str)

Parses the given server_string and returns a list of host and port. If it’s not a combination of host part and port, the port element is a null string. If the input is invalid expression, return a null list.

parse_strtime(timestr, fmt='%Y-%m-%dT%H:%M:%S.%f')

Turn a formatted time back into a datetime.

partition_dict(dict_, keys)

Return two dicts, one with keys the other with everything else.

read_cached_file(filename, cache_info, reload_func=None)

Read from a file if it has been modified.

Parameters:
  • cache_info – dictionary to hold opaque cache.
  • reload_func – optional function to be called with data when file is reloaded due to a modification.
Returns:

data from file

read_file_as_root(file_path)

Secure helper to read file as root.

safe_minidom_parse_string(xml_string)

Parse an XML string using minidom safely.

sanitize_hostname(hostname)

Return a hostname which conforms to RFC-952 and RFC-1123 specs.

save_and_reraise_exception(*args, **kwds)

Save current exception, run some code and then re-raise.

In some cases the exception context can be cleared, resulting in None being attempted to be reraised after an exception handler is run. This can happen when eventlet switches greenthreads or when running an exception handler, code raises and catches an exception. In both cases the exception context will be cleared.

To work around this, we save the exception state, run handler code, and then re-raise the original exception. If another exception occurs, the saved exception is logged and the new exception is reraised.

service_is_up(service)

Check whether a service is up based on last heartbeat.

set_time_override(override_time=datetime.datetime(2013, 2, 19, 20, 16, 29, 476953))

Override utils.utcnow to return a constant time.

ssh_execute(ssh, cmd, process_input=None, addl_env=None, check_exit_code=True)
str_dict_replace(s, mapping)
strcmp_const_time(s1, s2)

Constant-time string comparison.

Params s1:the first string
Params s2:the second string
Returns:True if the strings are equal.

This function takes two strings and compares them. It is intended to be used when doing a comparison for authentication purposes to help guard against timing attacks.

strtime(at=None, fmt='%Y-%m-%dT%H:%M:%S.%f')

Returns formatted utcnow.

subset_dict(dict_, keys)

Return a dict that only contains a subset of keys.

synchronized(name, external=False)

Synchronization decorator.

Decorating a method like so:

@synchronized('mylock')
def foo(self, *args):
   ...

ensures that only one thread will execute the bar method at a time.

Different methods can share the same lock:

@synchronized('mylock')
def foo(self, *args):
   ...

@synchronized('mylock')
def bar(self, *args):
   ...

This way only one of either foo or bar can be executing at a time.

The external keyword argument denotes whether this lock should work across multiple processes. This means that if two different workers both run a a method decorated with @synchronized(‘mylock’, external=True), only one of them will execute at a time.

Important limitation: you can only have one external lock running per thread at a time. For example the following will fail:

@utils.synchronized(‘testlock1’, external=True) def outer_lock():

@utils.synchronized(‘testlock2’, external=True) def inner_lock():

pass

inner_lock()

outer_lock()

tempdir(*args, **kwds)
temporary_chown(*args, **kwds)

Temporarily chown a path.

Params owner_uid:
 UID of temporary owner (defaults to current user)
temporary_mutation(*args, **kwds)

Temporarily set the attr on a particular object to a given value then revert when finished.

One use of this is to temporarily set the read_deleted flag on a context object:

with temporary_mutation(context, read_deleted=”yes”):
do_something_that_needed_deleted_objects()
timefunc(func)

Decorator that logs how long a particular function took to execute

to_primitive(value, convert_instances=False, level=0)

Convert a complex object into primitives.

Handy for JSON serialization. We can optionally handle instances, but since this is a recursive function, we could have cyclical data structures.

To handle cyclical data structures we could track the actual objects visited in a set, but not all objects are hashable. Instead we just track the depth of the object inspections and don’t go too deep.

Therefore, convert_instances=True is lossy ... be aware.

total_seconds(td)

Local total_seconds implementation for compatibility with python 2.6

trycmd(*args, **kwargs)

A wrapper around execute() to more easily handle warnings and errors.

Returns an (out, err) tuple of strings containing the output of the command’s stdout and stderr. If ‘err’ is not empty then the command can be considered to have failed.

:discard_warnings True | False. Defaults to False. If set to True,
then for succeeding commands, stderr is cleared
usage_from_instance(instance_ref, network_info=None, **kw)
utcnow()

Overridable version of utils.utcnow.

utcnow_ts()

Timestamp version of our utcnow function.

utf8(value)

Try to turn a string into utf-8 if possible.

Code is directly from the utf8 function in http://github.com/facebook/tornado/blob/master/tornado/escape.py

vpn_ping(address, port, timeout=0.05, session_id=None)

Sends a vpn negotiation packet and returns the server session.

Returns False on a failure. Basic packet structure is below.

Client packet (14 bytes):

 0 1      8 9  13
+-+--------+-----+
|x| cli_id |?????|
+-+--------+-----+
x = packet identifier 0x38
cli_id = 64 bit identifier
? = unknown, probably flags/padding

Server packet (26 bytes):

 0 1      8 9  13 14    21 2225
+-+--------+-----+--------+----+
|x| srv_id |?????| cli_id |????|
+-+--------+-----+--------+----+
x = packet identifier 0x40
cli_id = 64 bit identifier
? = unknown, probably flags/padding
bit 9 was 1 and the rest were 0 in testing
walk_class_hierarchy(clazz, encountered=None)

Walk class hierarchy, yielding most derived classes first

warn_deprecated_class(cls, msg)

Issues a warning to indicate that the given class is deprecated. If a message is given, it is appended to the deprecation warning.

warn_deprecated_function(func, msg)

Issues a warning to indicate that the given function is deprecated. If a message is given, it is appended to the deprecation warning.

xhtml_escape(value)

Escapes a string so it is valid within XML or XHTML.