formats.txt 31 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682
  1. =====================================
  2. The Internal Structure of Python Eggs
  3. =====================================
  4. STOP! This is not the first document you should read!
  5. .. contents:: **Table of Contents**
  6. ----------------------
  7. Eggs and their Formats
  8. ----------------------
  9. A "Python egg" is a logical structure embodying the release of a
  10. specific version of a Python project, comprising its code, resources,
  11. and metadata. There are multiple formats that can be used to physically
  12. encode a Python egg, and others can be developed. However, a key
  13. principle of Python eggs is that they should be discoverable and
  14. importable. That is, it should be possible for a Python application to
  15. easily and efficiently find out what eggs are present on a system, and
  16. to ensure that the desired eggs' contents are importable.
  17. There are two basic formats currently implemented for Python eggs:
  18. 1. ``.egg`` format: a directory or zipfile *containing* the project's
  19. code and resources, along with an ``EGG-INFO`` subdirectory that
  20. contains the project's metadata
  21. 2. ``.egg-info`` format: a file or directory placed *adjacent* to the
  22. project's code and resources, that directly contains the project's
  23. metadata.
  24. Both formats can include arbitrary Python code and resources, including
  25. static data files, package and non-package directories, Python
  26. modules, C extension modules, and so on. But each format is optimized
  27. for different purposes.
  28. The ``.egg`` format is well-suited to distribution and the easy
  29. uninstallation or upgrades of code, since the project is essentially
  30. self-contained within a single directory or file, unmingled with any
  31. other projects' code or resources. It also makes it possible to have
  32. multiple versions of a project simultaneously installed, such that
  33. individual programs can select the versions they wish to use.
  34. The ``.egg-info`` format, on the other hand, was created to support
  35. backward-compatibility, performance, and ease of installation for system
  36. packaging tools that expect to install all projects' code and resources
  37. to a single directory (e.g. ``site-packages``). Placing the metadata
  38. in that same directory simplifies the installation process, since it
  39. isn't necessary to create ``.pth`` files or otherwise modify
  40. ``sys.path`` to include each installed egg.
  41. Its disadvantage, however, is that it provides no support for clean
  42. uninstallation or upgrades, and of course only a single version of a
  43. project can be installed to a given directory. Thus, support from a
  44. package management tool is required. (This is why setuptools' "install"
  45. command refers to this type of egg installation as "single-version,
  46. externally managed".) Also, they lack sufficient data to allow them to
  47. be copied from their installation source. easy_install can "ship" an
  48. application by copying ``.egg`` files or directories to a target
  49. location, but it cannot do this for ``.egg-info`` installs, because
  50. there is no way to tell what code and resources belong to a particular
  51. egg -- there may be several eggs "scrambled" together in a single
  52. installation location, and the ``.egg-info`` format does not currently
  53. include a way to list the files that were installed. (This may change
  54. in a future version.)
  55. Code and Resources
  56. ==================
  57. The layout of the code and resources is dictated by Python's normal
  58. import layout, relative to the egg's "base location".
  59. For the ``.egg`` format, the base location is the ``.egg`` itself. That
  60. is, adding the ``.egg`` filename or directory name to ``sys.path``
  61. makes its contents importable.
  62. For the ``.egg-info`` format, however, the base location is the
  63. directory that *contains* the ``.egg-info``, and thus it is the
  64. directory that must be added to ``sys.path`` to make the egg importable.
  65. (Note that this means that the "normal" installation of a package to a
  66. ``sys.path`` directory is sufficient to make it an "egg" if it has an
  67. ``.egg-info`` file or directory installed alongside of it.)
  68. Project Metadata
  69. =================
  70. If eggs contained only code and resources, there would of course be
  71. no difference between them and any other directory or zip file on
  72. ``sys.path``. Thus, metadata must also be included, using a metadata
  73. file or directory.
  74. For the ``.egg`` format, the metadata is placed in an ``EGG-INFO``
  75. subdirectory, directly within the ``.egg`` file or directory. For the
  76. ``.egg-info`` format, metadata is stored directly within the
  77. ``.egg-info`` directory itself.
  78. The minimum project metadata that all eggs must have is a standard
  79. Python ``PKG-INFO`` file, named ``PKG-INFO`` and placed within the
  80. metadata directory appropriate to the format. Because it's possible for
  81. this to be the only metadata file included, ``.egg-info`` format eggs
  82. are not required to be a directory; they can just be a ``.egg-info``
  83. file that directly contains the ``PKG-INFO`` metadata. This eliminates
  84. the need to create a directory just to store one file. This option is
  85. *not* available for ``.egg`` formats, since setuptools always includes
  86. other metadata. (In fact, setuptools itself never generates
  87. ``.egg-info`` files, either; the support for using files was added so
  88. that the requirement could easily be satisfied by other tools, such
  89. as the distutils in Python 2.5).
  90. In addition to the ``PKG-INFO`` file, an egg's metadata directory may
  91. also include files and directories representing various forms of
  92. optional standard metadata (see the section on `Standard Metadata`_,
  93. below) or user-defined metadata required by the project. For example,
  94. some projects may define a metadata format to describe their application
  95. plugins, and metadata in this format would then be included by plugin
  96. creators in their projects' metadata directories.
  97. Filename-Embedded Metadata
  98. ==========================
  99. To allow introspection of installed projects and runtime resolution of
  100. inter-project dependencies, a certain amount of information is embedded
  101. in egg filenames. At a minimum, this includes the project name, and
  102. ideally will also include the project version number. Optionally, it
  103. can also include the target Python version and required runtime
  104. platform if platform-specific C code is included. The syntax of an
  105. egg filename is as follows::
  106. name ["-" version ["-py" pyver ["-" required_platform]]] "." ext
  107. The "name" and "version" should be escaped using the ``to_filename()``
  108. function provided by ``pkg_resources``, after first processing them with
  109. ``safe_name()`` and ``safe_version()`` respectively. These latter two
  110. functions can also be used to later "unescape" these parts of the
  111. filename. (For a detailed description of these transformations, please
  112. see the "Parsing Utilities" section of the ``pkg_resources`` manual.)
  113. The "pyver" string is the Python major version, as found in the first
  114. 3 characters of ``sys.version``. "required_platform" is essentially
  115. a distutils ``get_platform()`` string, but with enhancements to properly
  116. distinguish Mac OS versions. (See the ``get_build_platform()``
  117. documentation in the "Platform Utilities" section of the
  118. ``pkg_resources`` manual for more details.)
  119. Finally, the "ext" is either ``.egg`` or ``.egg-info``, as appropriate
  120. for the egg's format.
  121. Normally, an egg's filename should include at least the project name and
  122. version, as this allows the runtime system to find desired project
  123. versions without having to read the egg's PKG-INFO to determine its
  124. version number.
  125. Setuptools, however, only includes the version number in the filename
  126. when an ``.egg`` file is built using the ``bdist_egg`` command, or when
  127. an ``.egg-info`` directory is being installed by the
  128. ``install_egg_info`` command. When generating metadata for use with the
  129. original source tree, it only includes the project name, so that the
  130. directory will not have to be renamed each time the project's version
  131. changes.
  132. This is especially important when version numbers change frequently, and
  133. the source metadata directory is kept under version control with the
  134. rest of the project. (As would be the case when the project's source
  135. includes project-defined metadata that is not generated from by
  136. setuptools from data in the setup script.)
  137. Egg Links
  138. =========
  139. In addition to the ``.egg`` and ``.egg-info`` formats, there is a third
  140. egg-related extension that you may encounter on occasion: ``.egg-link``
  141. files.
  142. These files are not eggs, strictly speaking. They simply provide a way
  143. to reference an egg that is not physically installed in the desired
  144. location. They exist primarily as a cross-platform alternative to
  145. symbolic links, to support "installing" code that is being developed in
  146. a different location than the desired installation location. For
  147. example, if a user is developing an application plugin in their home
  148. directory, but the plugin needs to be "installed" in an application
  149. plugin directory, running "setup.py develop -md /path/to/app/plugins"
  150. will install an ``.egg-link`` file in ``/path/to/app/plugins``, that
  151. tells the egg runtime system where to find the actual egg (the user's
  152. project source directory and its ``.egg-info`` subdirectory).
  153. ``.egg-link`` files are named following the format for ``.egg`` and
  154. ``.egg-info`` names, but only the project name is included; no version,
  155. Python version, or platform information is included. When the runtime
  156. searches for available eggs, ``.egg-link`` files are opened and the
  157. actual egg file/directory name is read from them.
  158. Each ``.egg-link`` file should contain a single file or directory name,
  159. with no newlines. This filename should be the base location of one or
  160. more eggs. That is, the name must either end in ``.egg``, or else it
  161. should be the parent directory of one or more ``.egg-info`` format eggs.
  162. As of setuptools 0.6c6, the path may be specified as a platform-independent
  163. (i.e. ``/``-separated) relative path from the directory containing the
  164. ``.egg-link`` file, and a second line may appear in the file, specifying a
  165. platform-independent relative path from the egg's base directory to its
  166. setup script directory. This allows installation tools such as EasyInstall
  167. to find the project's setup directory and build eggs or perform other setup
  168. commands on it.
  169. -----------------
  170. Standard Metadata
  171. -----------------
  172. In addition to the minimum required ``PKG-INFO`` metadata, projects can
  173. include a variety of standard metadata files or directories, as
  174. described below. Except as otherwise noted, these files and directories
  175. are automatically generated by setuptools, based on information supplied
  176. in the setup script or through analysis of the project's code and
  177. resources.
  178. Most of these files and directories are generated via "egg-info
  179. writers" during execution of the setuptools ``egg_info`` command, and
  180. are listed in the ``egg_info.writers`` entry point group defined by
  181. setuptools' own ``setup.py`` file.
  182. Project authors can register their own metadata writers as entry points
  183. in this group (as described in the setuptools manual under "Adding new
  184. EGG-INFO Files") to cause setuptools to generate project-specific
  185. metadata files or directories during execution of the ``egg_info``
  186. command. It is up to project authors to document these new metadata
  187. formats, if they create any.
  188. ``.txt`` File Formats
  189. =====================
  190. Files described in this section that have ``.txt`` extensions have a
  191. simple lexical format consisting of a sequence of text lines, each line
  192. terminated by a linefeed character (regardless of platform). Leading
  193. and trailing whitespace on each line is ignored, as are blank lines and
  194. lines whose first nonblank character is a ``#`` (comment symbol). (This
  195. is the parsing format defined by the ``yield_lines()`` function of
  196. the ``pkg_resources`` module.)
  197. All ``.txt`` files defined by this section follow this format, but some
  198. are also "sectioned" files, meaning that their contents are divided into
  199. sections, using square-bracketed section headers akin to Windows
  200. ``.ini`` format. Note that this does *not* imply that the lines within
  201. the sections follow an ``.ini`` format, however. Please see an
  202. individual metadata file's documentation for a description of what the
  203. lines and section names mean in that particular file.
  204. Sectioned files can be parsed using the ``split_sections()`` function;
  205. see the "Parsing Utilities" section of the ``pkg_resources`` manual for
  206. for details.
  207. Dependency Metadata
  208. ===================
  209. ``requires.txt``
  210. ----------------
  211. This is a "sectioned" text file. Each section is a sequence of
  212. "requirements", as parsed by the ``parse_requirements()`` function;
  213. please see the ``pkg_resources`` manual for the complete requirement
  214. parsing syntax.
  215. The first, unnamed section (i.e., before the first section header) in
  216. this file is the project's core requirements, which must be installed
  217. for the project to function. (Specified using the ``install_requires``
  218. keyword to ``setup()``).
  219. The remaining (named) sections describe the project's "extra"
  220. requirements, as specified using the ``extras_require`` keyword to
  221. ``setup()``. The section name is the name of the optional feature, and
  222. the section body lists that feature's dependencies.
  223. Note that it is not normally necessary to inspect this file directly;
  224. ``pkg_resources.Distribution`` objects have a ``requires()`` method
  225. that can be used to obtain ``Requirement`` objects describing the
  226. project's core and optional dependencies.
  227. ``setup_requires.txt``
  228. ----------------------
  229. Much like ``requires.txt`` except represents the requirements
  230. specified by the ``setup_requires`` parameter to the Distribution.
  231. ``dependency_links.txt``
  232. ------------------------
  233. A list of dependency URLs, one per line, as specified using the
  234. ``dependency_links`` keyword to ``setup()``. These may be direct
  235. download URLs, or the URLs of web pages containing direct download
  236. links, and will be used by EasyInstall to find dependencies, as though
  237. the user had manually provided them via the ``--find-links`` command
  238. line option. Please see the setuptools manual and EasyInstall manual
  239. for more information on specifying this option, and for information on
  240. how EasyInstall processes ``--find-links`` URLs.
  241. ``depends.txt`` -- Obsolete, do not create!
  242. -------------------------------------------
  243. This file follows an identical format to ``requires.txt``, but is
  244. obsolete and should not be used. The earliest versions of setuptools
  245. required users to manually create and maintain this file, so the runtime
  246. still supports reading it, if it exists. The new filename was created
  247. so that it could be automatically generated from ``setup()`` information
  248. without overwriting an existing hand-created ``depends.txt``, if one
  249. was already present in the project's source ``.egg-info`` directory.
  250. ``namespace_packages.txt`` -- Namespace Package Metadata
  251. ========================================================
  252. A list of namespace package names, one per line, as supplied to the
  253. ``namespace_packages`` keyword to ``setup()``. Please see the manuals
  254. for setuptools and ``pkg_resources`` for more information about
  255. namespace packages.
  256. ``entry_points.txt`` -- "Entry Point"/Plugin Metadata
  257. =====================================================
  258. This is a "sectioned" text file, whose contents encode the
  259. ``entry_points`` keyword supplied to ``setup()``. All sections are
  260. named, as the section names specify the entry point groups in which the
  261. corresponding section's entry points are registered.
  262. Each section is a sequence of "entry point" lines, each parseable using
  263. the ``EntryPoint.parse`` classmethod; please see the ``pkg_resources``
  264. manual for the complete entry point parsing syntax.
  265. Note that it is not necessary to parse this file directly; the
  266. ``pkg_resources`` module provides a variety of APIs to locate and load
  267. entry points automatically. Please see the setuptools and
  268. ``pkg_resources`` manuals for details on the nature and uses of entry
  269. points.
  270. The ``scripts`` Subdirectory
  271. ============================
  272. This directory is currently only created for ``.egg`` files built by
  273. the setuptools ``bdist_egg`` command. It will contain copies of all
  274. of the project's "traditional" scripts (i.e., those specified using the
  275. ``scripts`` keyword to ``setup()``). This is so that they can be
  276. reconstituted when an ``.egg`` file is installed.
  277. The scripts are placed here using the distutils' standard
  278. ``install_scripts`` command, so any ``#!`` lines reflect the Python
  279. installation where the egg was built. But instead of copying the
  280. scripts to the local script installation directory, EasyInstall writes
  281. short wrapper scripts that invoke the original scripts from inside the
  282. egg, after ensuring that sys.path includes the egg and any eggs it
  283. depends on. For more about `script wrappers`_, see the section below on
  284. `Installation and Path Management Issues`_.
  285. Zip Support Metadata
  286. ====================
  287. ``native_libs.txt``
  288. -------------------
  289. A list of C extensions and other dynamic link libraries contained in
  290. the egg, one per line. Paths are ``/``-separated and relative to the
  291. egg's base location.
  292. This file is generated as part of ``bdist_egg`` processing, and as such
  293. only appears in ``.egg`` files (and ``.egg`` directories created by
  294. unpacking them). It is used to ensure that all libraries are extracted
  295. from a zipped egg at the same time, in case there is any direct linkage
  296. between them. Please see the `Zip File Issues`_ section below for more
  297. information on library and resource extraction from ``.egg`` files.
  298. ``eager_resources.txt``
  299. -----------------------
  300. A list of resource files and/or directories, one per line, as specified
  301. via the ``eager_resources`` keyword to ``setup()``. Paths are
  302. ``/``-separated and relative to the egg's base location.
  303. Resource files or directories listed here will be extracted
  304. simultaneously, if any of the named resources are extracted, or if any
  305. native libraries listed in ``native_libs.txt`` are extracted. Please
  306. see the setuptools manual for details on what this feature is used for
  307. and how it works, as well as the `Zip File Issues`_ section below.
  308. ``zip-safe`` and ``not-zip-safe``
  309. ---------------------------------
  310. These are zero-length files, and either one or the other should exist.
  311. If ``zip-safe`` exists, it means that the project will work properly
  312. when installed as an ``.egg`` zipfile, and conversely the existence of
  313. ``not-zip-safe`` means the project should not be installed as an
  314. ``.egg`` file. The ``zip_safe`` option to setuptools' ``setup()``
  315. determines which file will be written. If the option isn't provided,
  316. setuptools attempts to make its own assessment of whether the package
  317. can work, based on code and content analysis.
  318. If neither file is present at installation time, EasyInstall defaults
  319. to assuming that the project should be unzipped. (Command-line options
  320. to EasyInstall, however, take precedence even over an existing
  321. ``zip-safe`` or ``not-zip-safe`` file.)
  322. Note that these flag files appear only in ``.egg`` files generated by
  323. ``bdist_egg``, and in ``.egg`` directories created by unpacking such an
  324. ``.egg`` file.
  325. ``top_level.txt`` -- Conflict Management Metadata
  326. =================================================
  327. This file is a list of the top-level module or package names provided
  328. by the project, one Python identifier per line.
  329. Subpackages are not included; a project containing both a ``foo.bar``
  330. and a ``foo.baz`` would include only one line, ``foo``, in its
  331. ``top_level.txt``.
  332. This data is used by ``pkg_resources`` at runtime to issue a warning if
  333. an egg is added to ``sys.path`` when its contained packages may have
  334. already been imported.
  335. (It was also once used to detect conflicts with non-egg packages at
  336. installation time, but in more recent versions, setuptools installs eggs
  337. in such a way that they always override non-egg packages, thus
  338. preventing a problem from arising.)
  339. ``SOURCES.txt`` -- Source Files Manifest
  340. ========================================
  341. This file is roughly equivalent to the distutils' ``MANIFEST`` file.
  342. The differences are as follows:
  343. * The filenames always use ``/`` as a path separator, which must be
  344. converted back to a platform-specific path whenever they are read.
  345. * The file is automatically generated by setuptools whenever the
  346. ``egg_info`` or ``sdist`` commands are run, and it is *not*
  347. user-editable.
  348. Although this metadata is included with distributed eggs, it is not
  349. actually used at runtime for any purpose. Its function is to ensure
  350. that setuptools-built *source* distributions can correctly discover
  351. what files are part of the project's source, even if the list had been
  352. generated using revision control metadata on the original author's
  353. system.
  354. In other words, ``SOURCES.txt`` has little or no runtime value for being
  355. included in distributed eggs, and it is possible that future versions of
  356. the ``bdist_egg`` and ``install_egg_info`` commands will strip it before
  357. installation or distribution. Therefore, do not rely on its being
  358. available outside of an original source directory or source
  359. distribution.
  360. ------------------------------
  361. Other Technical Considerations
  362. ------------------------------
  363. Zip File Issues
  364. ===============
  365. Although zip files resemble directories, they are not fully
  366. substitutable for them. Most platforms do not support loading dynamic
  367. link libraries contained in zipfiles, so it is not possible to directly
  368. import C extensions from ``.egg`` zipfiles. Similarly, there are many
  369. existing libraries -- whether in Python or C -- that require actual
  370. operating system filenames, and do not work with arbitrary "file-like"
  371. objects or in-memory strings, and thus cannot operate directly on the
  372. contents of zip files.
  373. To address these issues, the ``pkg_resources`` module provides a
  374. "resource API" to support obtaining either the contents of a resource,
  375. or a true operating system filename for the resource. If the egg
  376. containing the resource is a directory, the resource's real filename
  377. is simply returned. However, if the egg is a zipfile, then the
  378. resource is first extracted to a cache directory, and the filename
  379. within the cache is returned.
  380. The cache directory is determined by the ``pkg_resources`` API; please
  381. see the ``set_cache_path()`` and ``get_default_cache()`` documentation
  382. for details.
  383. The Extraction Process
  384. ----------------------
  385. Resources are extracted to a cache subdirectory whose name is based
  386. on the enclosing ``.egg`` filename and the path to the resource. If
  387. there is already a file of the correct name, size, and timestamp, its
  388. filename is returned to the requester. Otherwise, the desired file is
  389. extracted first to a temporary name generated using
  390. ``mkstemp(".$extract",target_dir)``, and then its timestamp is set to
  391. match the one in the zip file, before renaming it to its final name.
  392. (Some collision detection and resolution code is used to handle the
  393. fact that Windows doesn't overwrite files when renaming.)
  394. If a resource directory is requested, all of its contents are
  395. recursively extracted in this fashion, to ensure that the directory
  396. name can be used as if it were valid all along.
  397. If the resource requested for extraction is listed in the
  398. ``native_libs.txt`` or ``eager_resources.txt`` metadata files, then
  399. *all* resources listed in *either* file will be extracted before the
  400. requested resource's filename is returned, thus ensuring that all
  401. C extensions and data used by them will be simultaneously available.
  402. Extension Import Wrappers
  403. -------------------------
  404. Since Python's built-in zip import feature does not support loading
  405. C extension modules from zipfiles, the setuptools ``bdist_egg`` command
  406. generates special import wrappers to make it work.
  407. The wrappers are ``.py`` files (along with corresponding ``.pyc``
  408. and/or ``.pyo`` files) that have the same module name as the
  409. corresponding C extension. These wrappers are located in the same
  410. package directory (or top-level directory) within the zipfile, so that
  411. say, ``foomodule.so`` will get a corresponding ``foo.py``, while
  412. ``bar/baz.pyd`` will get a corresponding ``bar/baz.py``.
  413. These wrapper files contain a short stanza of Python code that asks
  414. ``pkg_resources`` for the filename of the corresponding C extension,
  415. then reloads the module using the obtained filename. This will cause
  416. ``pkg_resources`` to first ensure that all of the egg's C extensions
  417. (and any accompanying "eager resources") are extracted to the cache
  418. before attempting to link to the C library.
  419. Note, by the way, that ``.egg`` directories will also contain these
  420. wrapper files. However, Python's default import priority is such that
  421. C extensions take precedence over same-named Python modules, so the
  422. import wrappers are ignored unless the egg is a zipfile.
  423. Installation and Path Management Issues
  424. =======================================
  425. Python's initial setup of ``sys.path`` is very dependent on the Python
  426. version and installation platform, as well as how Python was started
  427. (i.e., script vs. ``-c`` vs. ``-m`` vs. interactive interpreter).
  428. In fact, Python also provides only two relatively robust ways to affect
  429. ``sys.path`` outside of direct manipulation in code: the ``PYTHONPATH``
  430. environment variable, and ``.pth`` files.
  431. However, with no cross-platform way to safely and persistently change
  432. environment variables, this leaves ``.pth`` files as EasyInstall's only
  433. real option for persistent configuration of ``sys.path``.
  434. But ``.pth`` files are rather strictly limited in what they are allowed
  435. to do normally. They add directories only to the *end* of ``sys.path``,
  436. after any locally-installed ``site-packages`` directory, and they are
  437. only processed *in* the ``site-packages`` directory to start with.
  438. This is a double whammy for users who lack write access to that
  439. directory, because they can't create a ``.pth`` file that Python will
  440. read, and even if a sympathetic system administrator adds one for them
  441. that calls ``site.addsitedir()`` to allow some other directory to
  442. contain ``.pth`` files, they won't be able to install newer versions of
  443. anything that's installed in the systemwide ``site-packages``, because
  444. their paths will still be added *after* ``site-packages``.
  445. So EasyInstall applies two workarounds to solve these problems.
  446. The first is that EasyInstall leverages ``.pth`` files' "import" feature
  447. to manipulate ``sys.path`` and ensure that anything EasyInstall adds
  448. to a ``.pth`` file will always appear before both the standard library
  449. and the local ``site-packages`` directories. Thus, it is always
  450. possible for a user who can write a Python-read ``.pth`` file to ensure
  451. that their packages come first in their own environment.
  452. Second, when installing to a ``PYTHONPATH`` directory (as opposed to
  453. a "site" directory like ``site-packages``) EasyInstall will also install
  454. a special version of the ``site`` module. Because it's in a
  455. ``PYTHONPATH`` directory, this module will get control before the
  456. standard library version of ``site`` does. It will record the state of
  457. ``sys.path`` before invoking the "real" ``site`` module, and then
  458. afterwards it processes any ``.pth`` files found in ``PYTHONPATH``
  459. directories, including all the fixups needed to ensure that eggs always
  460. appear before the standard library in sys.path, but are in a relative
  461. order to one another that is defined by their ``PYTHONPATH`` and
  462. ``.pth``-prescribed sequence.
  463. The net result of these changes is that ``sys.path`` order will be
  464. as follows at runtime:
  465. 1. The ``sys.argv[0]`` directory, or an empty string if no script
  466. is being executed.
  467. 2. All eggs installed by EasyInstall in any ``.pth`` file in each
  468. ``PYTHONPATH`` directory, in order first by ``PYTHONPATH`` order,
  469. then normal ``.pth`` processing order (which is to say alphabetical
  470. by ``.pth`` filename, then by the order of listing within each
  471. ``.pth`` file).
  472. 3. All eggs installed by EasyInstall in any ``.pth`` file in each "site"
  473. directory (such as ``site-packages``), following the same ordering
  474. rules as for the ones on ``PYTHONPATH``.
  475. 4. The ``PYTHONPATH`` directories themselves, in their original order
  476. 5. Any paths from ``.pth`` files found on ``PYTHONPATH`` that were *not*
  477. eggs installed by EasyInstall, again following the same relative
  478. ordering rules.
  479. 6. The standard library and "site" directories, along with the contents
  480. of any ``.pth`` files found in the "site" directories.
  481. Notice that sections 1, 4, and 6 comprise the "normal" Python setup for
  482. ``sys.path``. Sections 2 and 3 are inserted to support eggs, and
  483. section 5 emulates what the "normal" semantics of ``.pth`` files on
  484. ``PYTHONPATH`` would be if Python natively supported them.
  485. For further discussion of the tradeoffs that went into this design, as
  486. well as notes on the actual magic inserted into ``.pth`` files to make
  487. them do these things, please see also the following messages to the
  488. distutils-SIG mailing list:
  489. * http://mail.python.org/pipermail/distutils-sig/2006-February/006026.html
  490. * http://mail.python.org/pipermail/distutils-sig/2006-March/006123.html
  491. Script Wrappers
  492. ---------------
  493. EasyInstall never directly installs a project's original scripts to
  494. a script installation directory. Instead, it writes short wrapper
  495. scripts that first ensure that the project's dependencies are active
  496. on sys.path, before invoking the original script. These wrappers
  497. have a #! line that points to the version of Python that was used to
  498. install them, and their second line is always a comment that indicates
  499. the type of script wrapper, the project version required for the script
  500. to run, and information identifying the script to be invoked.
  501. The format of this marker line is::
  502. "# EASY-INSTALL-" script_type ": " tuple_of_strings "\n"
  503. The ``script_type`` is one of ``SCRIPT``, ``DEV-SCRIPT``, or
  504. ``ENTRY-SCRIPT``. The ``tuple_of_strings`` is a comma-separated
  505. sequence of Python string constants. For ``SCRIPT`` and ``DEV-SCRIPT``
  506. wrappers, there are two strings: the project version requirement, and
  507. the script name (as a filename within the ``scripts`` metadata
  508. directory). For ``ENTRY-SCRIPT`` wrappers, there are three:
  509. the project version requirement, the entry point group name, and the
  510. entry point name. (See the "Automatic Script Creation" section in the
  511. setuptools manual for more information about entry point scripts.)
  512. In each case, the project version requirement string will be a string
  513. parseable with the ``pkg_resources`` modules' ``Requirement.parse()``
  514. classmethod. The only difference between a ``SCRIPT`` wrapper and a
  515. ``DEV-SCRIPT`` is that a ``DEV-SCRIPT`` actually executes the original
  516. source script in the project's source tree, and is created when the
  517. "setup.py develop" command is run. A ``SCRIPT`` wrapper, on the other
  518. hand, uses the "installed" script written to the ``EGG-INFO/scripts``
  519. subdirectory of the corresponding ``.egg`` zipfile or directory.
  520. (``.egg-info`` eggs do not have script wrappers associated with them,
  521. except in the "setup.py develop" case.)
  522. The purpose of including the marker line in generated script wrappers is
  523. to facilitate introspection of installed scripts, and their relationship
  524. to installed eggs. For example, an uninstallation tool could use this
  525. data to identify what scripts can safely be removed, and/or identify
  526. what scripts would stop working if a particular egg is uninstalled.