[pull] main from python:main

Doc/library/csv.rst

@@ -324,14 +324,19 @@ The :mod:`!csv` module defines the following classes:
 
       If several combinations fit the sample equally well ---
       for example if both ``','`` and ``';'`` split every row consistently ---
-      the delimiters ``','``, ``'\t'``, ``';'``, ``' '`` and ``':'``
-      are preferred, in this order,
+      the delimiters listed in the :attr:`~Sniffer.preferred` attribute
+      are preferred, in that order,
       no matter how many times each of them occurs.
 
+      The *lineterminator* parameter is deduced separately,
+      by a majority vote among the line endings of the sample.
+      A tie is broken in the order ``'\r\n'``, ``'\n'``, ``'\r'``,
+      so a sample without a complete line gives ``'\r\n'``.
+
       .. versionchanged:: next
          The dialect is now deduced by trial parsing
          and the results may differ from those of earlier Python versions.
-         The *escapechar* parameter can now be detected,
+         The *escapechar* and *lineterminator* parameters can now be detected,
          and the requested *delimiters* are not restricted to ASCII.
 
 
@@ -354,6 +359,15 @@ The :mod:`!csv` module defines the following classes:
       This method is a rough heuristic and may produce both false positives and
       negatives.
 
+   The :class:`Sniffer` class has the following attribute:
+
+   .. attribute:: preferred
+
+      The list of the delimiters preferred for breaking ties,
+      in the order of preference.
+      It can be modified.
+      Its initial value is ``[',', '\t', ';', ' ', ':']``.
+
 An example for :class:`Sniffer` use::
 
    with open('example.csv', newline='') as csvfile:
@@ -377,6 +391,8 @@ The :mod:`!csv` module defines the following constants:
    Instructs :class:`writer` objects to only quote those fields which contain
    special characters such as *delimiter*, *quotechar*, ``'\r'``, ``'\n'``
    or any of the characters in *lineterminator*.
+   If *doublequote* is :const:`False` and *escapechar* is set,
+   the *quotechar* is escaped instead of causing the field to be quoted.
 
 
 .. data:: QUOTE_NONNUMERIC
@@ -481,6 +497,10 @@ Dialects support the following attributes:
    On reading, the *escapechar* removes any special meaning from
    the following character. It defaults to :const:`None`, which disables escaping.
 
+   .. versionchanged:: 3.10
+      Previously the *escapechar* itself was not escaped,
+      which lost it on reading.
+
    .. versionchanged:: 3.11
       An empty *escapechar* is not allowed.
 
@@ -528,6 +548,13 @@ Dialects support the following attributes:
    When ``True``, raise exception :exc:`Error` on bad CSV input.
    The default is ``False``.
 
+Dialects support :func:`copy.replace`,
+which returns a copy of the dialect
+with the specified formatting parameters replaced.
+
+.. versionchanged:: next
+   Added support for :func:`copy.replace`.
+
 .. _reader-objects:
 
 Reader Objects

Doc/library/curses.rst

@@ -31,6 +31,19 @@ Linux and the BSD variants of Unix.
    Whenever the documentation mentions a *character string* it can be specified
    as a Unicode string or a byte string.
 
+.. note::
+
+   Whether curses may be used from several threads
+   depends on the underlying library and how it was built.
+   In many implementations, including the default build of ncurses,
+   the screen state is shared and not thread-safe;
+   since the blocking and refresh methods
+   (such as :meth:`~window.getch` and :meth:`~window.refresh`)
+   release the :term:`GIL`,
+   unsynchronized use from several threads can then crash the interpreter.
+   Serialize the calls,
+   or wrap them in :meth:`window.use` and :meth:`screen.use`.
+
 .. seealso::
 
    Module :mod:`curses.ascii`

Doc/library/inspect.rst

@@ -747,6 +747,7 @@ Retrieving source code
 .. function:: getfile(object)
 
    Return the name of the (text or binary) file in which an object was defined.
+   An :exc:`OSError` is raised if the source code cannot be retrieved.
    This will fail with a :exc:`TypeError` if the object is a built-in module,
    class, or function.
 
@@ -760,9 +761,10 @@ Retrieving source code
 .. function:: getsourcefile(object)
 
    Return the name of the Python source file in which an object was defined
-   or ``None`` if no way can be identified to get the source.  This
-   will fail with a :exc:`TypeError` if the object is a built-in module, class, or
-   function.
+   or ``None`` if no way can be identified to get the source.  An :exc:`OSError` is
+   raised if the source code cannot be retrieved.
+   This will fail with a :exc:`TypeError` if the object is a built-in module,
+   class, or function.
 
 
 .. function:: getsourcelines(object)

Doc/library/optparse.rst

@@ -1073,6 +1073,12 @@ As you can see, most actions involve storing or updating a value somewhere.
    and can be overridden by a custom subclass passed to the *values* argument of
    :meth:`OptionParser.parse_args` (as described in :ref:`optparse-parsing-arguments`).
 
+   :class:`!Values` objects support :func:`copy.replace`,
+   which returns a copy of the object with the specified attributes replaced.
+
+   .. versionchanged:: next
+      Added support for :func:`copy.replace`.
+
 Option
 arguments (and various other values) are stored as attributes of this object,
 according to the :attr:`~Option.dest` (destination) option attribute.

Doc/library/os.path.rst

@@ -59,6 +59,18 @@ the :mod:`glob` module.)
    Return a normalized absolutized version of the pathname *path*. On most
    platforms, this is equivalent to calling ``normpath(join(os.getcwd(), path))``.
 
+   On Windows the path is normalized by the operating system,
+   therefore the result can differ from ``normpath(join(os.getcwd(), path))``.
+   A drive-relative path is resolved against the current directory
+   of the specified drive, and the drive letter is capitalized.
+   Trailing dots and spaces are stripped.
+   For example::
+
+      >>> os.path.abspath('c:spam')
+      'C:\\Temp\\spam'
+      >>> os.path.abspath('c:/temp/spam. . .')
+      'c:\\temp\\spam'
+
    .. seealso:: :func:`os.path.join` and :func:`os.path.normpath`.
 
    .. versionchanged:: 3.6
@@ -435,6 +447,9 @@ the :mod:`glob` module.)
    links encountered in the path (if they are supported by the operating
    system). On Windows, this function will also resolve MS-DOS (also called 8.3)
    style names such as ``C:\\PROGRA~1`` to ``C:\\Program Files``.
+   The returned path uses the case reported by the operating system,
+   which can differ from the case of *path*,
+   in particular the drive letter is capitalized.
 
    By default, the path is evaluated up to the first component that does not
    exist, is a symlink loop, or whose evaluation raises :exc:`OSError`.

Doc/library/site.rst

@@ -460,7 +460,7 @@ Module contents
    .. versionadded:: 3.2
 
    .. versionchanged:: 3.3
-      Add the optional *prefixes* argument.
+      Added the optional *prefixes* parameter.
 
 
 .. function:: getuserbase()

Doc/library/stdtypes.rst

@@ -2590,7 +2590,8 @@ expression support in the :mod:`re` module).
 
    Return a list of the words in the string, using *sep* as the delimiter string.
    If *maxsplit* is given, at most *maxsplit* splits are done, the *rightmost*
-   ones.  If *sep* is not specified or ``None``, any whitespace string is a
+   ones.  If *sep* is not specified or ``None``, any
+   :meth:`whitespace <str.isspace>` string is a
    separator.  Except for splitting from the right, :meth:`rsplit` behaves like
    :meth:`split` which is described in detail below.
 
@@ -2651,7 +2652,8 @@ expression support in the :mod:`re` module).
       ['1', '2', '3<4']
 
    If *sep* is not specified or is ``None``, a different splitting algorithm is
-   applied: runs of consecutive whitespace are regarded as a single separator,
+   applied: runs of consecutive :meth:`whitespace <str.isspace>` are regarded
+   as a single separator,
    and the result will contain no empty strings at the start or end if the
    string has leading or trailing whitespace.  Consequently, splitting an empty
    string or a string consisting of just whitespace with a ``None`` separator
@@ -3915,7 +3917,8 @@ produce new objects.
    Return a copy of the sequence with specified leading bytes removed.  The
    *bytes* argument is a binary sequence specifying the set of byte values to
    be removed.  If omitted or ``None``, the *bytes* argument defaults
-   to removing ASCII whitespace.  The *bytes* argument is not a prefix;
+   to removing :meth:`ASCII whitespace <bytes.isspace>`.
+   The *bytes* argument is not a prefix;
    rather, all combinations of its values are stripped::
 
       >>> b'   spacious   '.lstrip()
@@ -3959,7 +3962,8 @@ produce new objects.
    Split the binary sequence into subsequences of the same type, using *sep*
    as the delimiter string. If *maxsplit* is given, at most *maxsplit* splits
    are done, the *rightmost* ones.  If *sep* is not specified or ``None``,
-   any subsequence consisting solely of ASCII whitespace is a separator.
+   any subsequence consisting solely of
+   :meth:`ASCII whitespace <bytes.isspace>` is a separator.
    Except for splitting from the right, :meth:`rsplit` behaves like
    :meth:`split` which is described in detail below.
 
@@ -3970,7 +3974,8 @@ produce new objects.
    Return a copy of the sequence with specified trailing bytes removed.  The
    *bytes* argument is a binary sequence specifying the set of byte values to
    be removed.  If omitted or ``None``, the *bytes* argument defaults to
-   removing ASCII whitespace.  The *bytes* argument is not a suffix; rather,
+   removing :meth:`ASCII whitespace <bytes.isspace>`.
+   The *bytes* argument is not a suffix; rather,
    all combinations of its values are stripped::
 
       >>> b'   spacious   '.rstrip()
@@ -4023,7 +4028,8 @@ produce new objects.
       [b'1', b'2', b'3<4']
 
    If *sep* is not specified or is ``None``, a different splitting algorithm
-   is applied: runs of consecutive ASCII whitespace are regarded as a single
+   is applied: runs of consecutive :meth:`ASCII whitespace <bytes.isspace>`
+   are regarded as a single
    separator, and the result will contain no empty strings at the start or
    end if the sequence has leading or trailing whitespace.  Consequently,
    splitting an empty sequence or a sequence consisting solely of ASCII
@@ -4046,7 +4052,8 @@ produce new objects.
    Return a copy of the sequence with specified leading and trailing bytes
    removed. The *bytes* argument is a binary sequence specifying the set of
    byte values to be removed.  If omitted or ``None``, the *bytes*
-   argument defaults to removing ASCII whitespace. The *bytes* argument is
+   argument defaults to removing :meth:`ASCII whitespace <bytes.isspace>`.
+   The *bytes* argument is
    not a prefix or suffix; rather, all combinations of its values are
    stripped::
 

Doc/library/tarfile.rst

@@ -963,6 +963,11 @@ A ``TarInfo`` object has the following public data attributes:
    If *deep* is false, the copy is shallow, i.e. ``pax_headers``
    and any custom attributes are shared with the original ``TarInfo`` object.
 
+   This method is also used by :func:`copy.replace`.
+
+   .. versionchanged:: next
+      Added support for :func:`copy.replace`.
+
 A :class:`TarInfo` object also provides some convenient query methods:
 
 

Doc/whatsnew/3.16.rst

@@ -135,6 +135,11 @@ csv
   The results may differ from those of earlier Python versions.
   (Contributed by Serhiy Storchaka in :gh:`83273`.)
 
+* :meth:`csv.Sniffer.sniff` now detects the *lineterminator* parameter
+  by a majority vote among the line endings of the sample,
+  instead of always returning ``'\r\n'``.
+  (Contributed by Serhiy Storchaka in :gh:`75008`.)
+
 curses
 ------
 

Include/internal/pycore_optimizer.h

@@ -206,7 +206,7 @@ typedef struct _PyExecutorObject {
 PyAPI_FUNC(_PyExecutorObject*) _Py_GetExecutor(PyCodeObject *code, int offset);
 
 int _Py_ExecutorInit(_PyExecutorObject *, const _PyBloomFilter *);
-void _Py_ExecutorDetach(_PyExecutorObject *);
+PyAPI_FUNC(void) _Py_ExecutorDetach(_PyExecutorObject *);
 PyAPI_FUNC(void) _Py_Executor_DependsOn(_PyExecutorObject *executor, void *obj);
 
 /* We use a bloomfilter with k = 6, m = 256

Lib/csv.py

@@ -82,6 +82,11 @@ class excel:
            "unix_dialect"]
 
 
+_dialect_attributes = frozenset({
+    'delimiter', 'quotechar', 'escapechar', 'doublequote',
+    'skipinitialspace', 'lineterminator', 'quoting', 'strict',
+})
+
 class Dialect:
     """Describe a CSV dialect.
 
@@ -113,6 +118,17 @@ def _validate(self):
             # Re-raise to get a traceback showing more user code.
             raise Error(str(e)) from None
 
+    def __replace__(self, /, **changes):
+        unexpected = changes.keys() - _dialect_attributes
+        if unexpected:
+            raise TypeError(f'__replace__() got an unexpected keyword '
+                            f'argument {min(unexpected)!r}')
+        new = object.__new__(self.__class__)
+        new.__dict__.update(self.__dict__)
+        new.__dict__.update(changes)
+        new._validate()
+        return new
+
 class excel(Dialect):
     """Describe the usual properties of Excel-generated CSV files."""
     delimiter = ','
@@ -373,9 +389,9 @@ def sniff(self, sample, delimiters=None):
 
         class dialect(Dialect):
             _name = "sniffed"
-            lineterminator = '\r\n'
             quoting = QUOTE_MINIMAL
 
+        dialect.lineterminator = self._detect_lineterminator(lines)
         dialect.delimiter = delimiter
         # _csv.reader won't accept a quotechar of ''
         dialect.quotechar = quotechar or '"'
@@ -598,6 +614,24 @@ def _detect_skipinitialspace(self, lines, delimiter, quotechar,
                  for kept_row, skipped_row in zip(*results)]
         return all(first) or not any(first)
 
+    def _detect_lineterminator(self, lines):
+        """
+        Detect the line terminator by majority vote among the line
+        endings.  A line break inside a quoted field is counted too,
+        but it takes more of them than of the real ones to win the
+        vote.  A tie is broken in the order '\\r\\n', '\\n', '\\r',
+        so a sample without a complete line gives '\\r\\n'.
+        """
+        counts = dict.fromkeys(('\r\n', '\n', '\r'), 0)
+        for line in lines:
+            for lineterminator in counts:
+                if line.endswith(lineterminator):
+                    counts[lineterminator] += 1
+                    break
+        # max() returns the first of equal candidates, and dict
+        # preserves the insertion order.
+        return max(counts, key=counts.get)
+
     def has_header(self, sample):
         # Creates a dictionary of types of data in each column. If any
         # column is of a single type (say, integers), *except* for the first

Lib/doctest.py

@@ -285,7 +285,8 @@ def getvalue(self):
         return result
 
     def truncate(self, size=None):
-        self.seek(size)
+        if size is not None:
+            self.seek(size)
         StringIO.truncate(self)
 
 # Worst-case linear-time ellipsis matching.
@@ -922,7 +923,7 @@ def find(self, obj, name=None, module=None, globs=None, extraglobs=None):
         # given object's docstring.
         try:
             file = inspect.getsourcefile(obj)
-        except TypeError:
+        except (TypeError, OSError):
             source_lines = None
         else:
             if not file:

Lib/ipaddress.py

@@ -1911,14 +1911,18 @@ def _explode_shorthand_ip_string(self):
         elif isinstance(self, IPv6Interface):
             ip_str = str(self.ip)
         else:
-            ip_str = str(self)
+            ip_str = self._string_from_ip_int(self._ip)
 
         ip_int = self._ip_int_from_string(ip_str)
         hex_str = '%032x' % ip_int
-        parts = [hex_str[x:x+4] for x in range(0, 32, 4)]
-        if isinstance(self, (_BaseNetwork, IPv6Interface)):
-            return '%s/%d' % (':'.join(parts), self._prefixlen)
-        return ':'.join(parts)
+        exploded = ':'.join([hex_str[x:x+4] for x in range(0, 32, 4)])
+        if isinstance(self, _BaseNetwork):
+            return '%s/%d' % (exploded, self._prefixlen)
+        if self._scope_id:
+            exploded = '%s%%%s' % (exploded, self._scope_id)
+        if isinstance(self, IPv6Interface):
+            return '%s/%d' % (exploded, self._prefixlen)
+        return exploded
 
     def _reverse_pointer(self):
         """Return the reverse DNS pointer name for the IPv6 address.

Lib/optparse.py

@@ -830,6 +830,12 @@ def __eq__(self, other):
         else:
             return NotImplemented
 
+    def __replace__(self, /, **changes):
+        new = self.__class__()
+        new.__dict__.update(self.__dict__)
+        new.__dict__.update(changes)
+        return new
+
     def _update_careful(self, dict):
         """
         Update the option values from an arbitrary dictionary, but only