1. 11 Dec, 2018 1 commit
  2. 27 Aug, 2018 1 commit
    • Arnd Bergmann's avatar
      y2038: globally rename compat_time to old_time32 · 9afc5eee
      Arnd Bergmann authored
      Christoph Hellwig suggested a slightly different path for handling
      backwards compatibility with the 32-bit time_t based system calls:
      Rather than simply reusing the compat_sys_* entry points on 32-bit
      architectures unchanged, we get rid of those entry points and the
      compat_time types by renaming them to something that makes more sense
      on 32-bit architectures (which don't have a compat mode otherwise),
      and then share the entry points under the new name with the 64-bit
      architectures that use them for implementing the compatibility.
      The following types and interfaces are renamed here, and moved
      from linux/compat_time.h to linux/time32.h:
      old				new
      ---				---
      compat_time_t			old_time32_t
      struct compat_timeval		struct old_timeval32
      struct compat_timespec		struct old_timespec32
      struct compat_itimerspec	struct old_itimerspec32
      ns_to_compat_timeval()		ns_to_old_timeval32()
      get_compat_itimerspec64()	get_old_itimerspec32()
      put_compat_itimerspec64()	put_old_itimer...
  3. 06 Aug, 2018 3 commits
  4. 18 Jul, 2018 1 commit
  5. 12 Jul, 2018 2 commits
  6. 28 Jun, 2018 1 commit
    • Linus Torvalds's avatar
      Revert changes to convert to ->poll_mask() and aio IOCB_CMD_POLL · a11e1d43
      Linus Torvalds authored
      The poll() changes were not well thought out, and completely
      unexplained.  They also caused a huge performance regression, because
      "->poll()" was no longer a trivial file operation that just called down
      to the underlying file operations, but instead did at least two indirect
      Indirect calls are sadly slow now with the Spectre mitigation, but the
      performance problem could at least be largely mitigated by changing the
      "->get_poll_head()" operation to just have a per-file-descriptor pointer
      to the poll head instead.  That gets rid of one of the new indirections.
      But that doesn't fix the new complexity that is completely unwarranted
      for the regular case.  The (undocumented) reason for the poll() changes
      was some alleged AIO poll race fixing, but we don't make the common case
      slower and more complex for some uncommon special case, so this all
      really needs way more explanations and most likely a fundamental
      [ This revert is a re...
  7. 15 Jun, 2018 1 commit
  8. 04 Jun, 2018 1 commit
  9. 31 May, 2018 2 commits
  10. 30 May, 2018 6 commits
  11. 28 May, 2018 1 commit
  12. 26 May, 2018 4 commits
    • Christoph Hellwig's avatar
      aio: try to complete poll iocbs without context switch · 1962da0d
      Christoph Hellwig authored
      If we can acquire ctx_lock without spinning we can just remove our
      iocb from the active_reqs list, and thus complete the iocbs from the
      wakeup context.
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
    • Christoph Hellwig's avatar
      aio: implement IOCB_CMD_POLL · 2c14fa83
      Christoph Hellwig authored
      Simple one-shot poll through the io_submit() interface.  To poll for
      a file descriptor the application should submit an iocb of type
      IOCB_CMD_POLL.  It will poll the fd for the events specified in the
      the first 32 bits of the aio_buf field of the iocb.
      Unlike poll or epoll without EPOLLONESHOT this interface always works
      in one shot mode, that is once the iocb is completed, it will have to be
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Reviewed-by: default avatarDarrick J. Wong <darrick.wong@oracle.com>
    • Christoph Hellwig's avatar
      aio: simplify cancellation · 888933f8
      Christoph Hellwig authored
      With the current aio code there is no need for the magic KIOCB_CANCELLED
      value, as a cancelation just kicks the driver to queue the completion
      ASAP, with all actual completion handling done in another thread. Given
      that both the completion path and cancelation take the context lock there
      is no need for magic cmpxchg loops either.  If we remove iocbs from the
      active list after calling ->ki_cancel (but with ctx_lock still held), we
      can also rely on the invariant thay anything found on the list has a
      ->ki_cancel callback and can be cancelled, further simplifing the code.
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
    • Christoph Hellwig's avatar
      aio: simplify KIOCB_KEY handling · f3a2752a
      Christoph Hellwig authored
      No need to pass the key field to lookup_iocb to compare it with KIOCB_KEY,
      as we can do that right after retrieving it from userspace.  Also move the
      KIOCB_KEY definition to aio.c as it is an internal value not used by any
      other place in the kernel.
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
  13. 24 May, 2018 1 commit
    • Al Viro's avatar
      fix io_destroy()/aio_complete() race · 4faa9996
      Al Viro authored
      If io_destroy() gets to cancelling everything that can be cancelled and
      gets to kiocb_cancel() calling the function driver has left in ->ki_cancel,
      it becomes vulnerable to a race with IO completion.  At that point req
      is already taken off the list and aio_complete() does *NOT* spin until
      we (in free_ioctx_users()) releases ->ctx_lock.  As the result, it proceeds
      to kiocb_free(), freing req just it gets passed to ->ki_cancel().
      Fix is simple - remove from the list after the call of kiocb_cancel().  All
      instances of ->ki_cancel() already have to cope with the being called with
      iocb still on list - that's what happens in io_cancel(2).
      Cc: stable@kernel.org
      Fixes: 0460fef2
       "aio: use cancellation list lazily"
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
  14. 21 May, 2018 1 commit
    • Al Viro's avatar
      aio: fix io_destroy(2) vs. lookup_ioctx() race · baf10564
      Al Viro authored
      kill_ioctx() used to have an explicit RCU delay between removing the
      reference from ->ioctx_table and percpu_ref_kill() dropping the refcount.
      At some point that delay had been removed, on the theory that
      percpu_ref_kill() itself contained an RCU delay.  Unfortunately, that was
      the wrong kind of RCU delay and it didn't care about rcu_read_lock() used
      by lookup_ioctx().  As the result, we could get ctx freed right under
      lookup_ioctx().  Tejun has fixed that in a6d7cff4 ("fs/aio: Add explicit
      RCU grace period when freeing kioctx"); however, that fix is not enough.
      Suppose io_destroy() from one thread races with e.g. io_setup() from another;
      CPU1 removes the reference from current->mm->ioctx_table[...] just as CPU2
      has picked it (under rcu_read_lock()).  Then CPU1 proceeds to drop the
      refcount, getting it to 0 and triggering a call of free_ioctx_users(),
      which proceeds to drop the secondary refcount and once that reaches zero
      calls free_ioctx_reqs().  That does
              INIT_RCU_WORK(&ctx->free_rwork, free_ioctx);
              queue_rcu_work(system_wq, &ctx->free_rwork);
      and schedules freeing the whole thing after RCU delay.
      In the meanwhile CPU2 has gotten around to percpu_ref_get(), bumping the
      refcount from 0 to 1 and returned the reference to io_setup().
      Tejun's fix (that queue_rcu_work() in there) guarantees that ctx won't get
      freed until after percpu_ref_get().  Sure, we'd increment the counter before
      ctx can be freed.  Now we are out of rcu_read_lock() and there's nothing to
      stop freeing of the whole thing.  Unfortunately, CPU2 assumes that since it
      has grabbed the reference, ctx is *NOT* going away until it gets around to
      dropping that reference.
      The fix is obvious - use percpu_ref_tryget_live() and treat failure as miss.
      It's not costlier than what we currently do in normal case, it's safe to
      call since freeing *is* delayed and it closes the race window - either
      lookup_ioctx() comes before percpu_ref_kill() (in which case ctx->users
      won't reach 0 until the caller of lookup_ioctx() drops it) or lookup_ioctx()
      fails, ctx->users is unaffected and caller of lookup_ioctx() doesn't see
      the object in question at all.
      Cc: stable@kernel.org
      Fixes: a6d7cff4
       "fs/aio: Add explicit RCU grace period when freeing kioctx"
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
  15. 02 May, 2018 7 commits
  16. 19 Mar, 2018 1 commit
  17. 14 Mar, 2018 2 commits
    • Tejun Heo's avatar
      fs/aio: Use RCU accessors for kioctx_table->table[] · d0264c01
      Tejun Heo authored
      While converting ioctx index from a list to a table, db446a08
      ("aio: convert the ioctx list to table lookup v3") missed tagging
      kioctx_table->table[] as an array of RCU pointers and using the
      appropriate RCU accessors.  This introduces a small window in the
      lookup path where init and access may race.
      Mark kioctx_table->table[] with __rcu and use the approriate RCU
      accessors when using the field.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Reported-by: default avatarJann Horn <jannh@google.com>
      Fixes: db446a08 ("aio: convert the ioctx list to table lookup v3")
      Cc: Benjamin LaHaise <bcrl@kvack.org>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: stable@vger.kernel.org # v3.12+
    • Tejun Heo's avatar
      fs/aio: Add explicit RCU grace period when freeing kioctx · a6d7cff4
      Tejun Heo authored
      While fixing refcounting, e34ecee2
       ("aio: Fix a trinity splat")
      incorrectly removed explicit RCU grace period before freeing kioctx.
      The intention seems to be depending on the internal RCU grace periods
      of percpu_ref; however, percpu_ref uses a different flavor of RCU,
      sched-RCU.  This can lead to kioctx being freed while RCU read
      protected dereferences are still in progress.
      Fix it by updating free_ioctx() to go through call_rcu() explicitly.
      v2: Comment added to explain double bouncing.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Reported-by: default avatarJann Horn <jannh@google.com>
      Fixes: e34ecee2 ("aio: Fix a trinity splat")
      Cc: Kent Overstreet <kent.overstreet@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: stable@vger.kernel.org # v3.13+
  18. 25 Oct, 2017 1 commit
    • Mark Rutland's avatar
      locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns... · 6aa7de05
      Mark Rutland authored
      locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE()
      Please do not apply this to mainline directly, instead please re-run the
      coccinelle script shown below and apply its output.
      For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
      preference to ACCESS_ONCE(), and new code is expected to use one of the
      former. So far, there's been no reason to change most existing uses of
      ACCESS_ONCE(), as these aren't harmful, and changing them results in
      However, for some features, the read/write distinction is critical to
      correct operation. To distinguish these cases, separate read/write
      accessors must be used. This patch migrates (most) remaining
      ACCESS_ONCE() instances to {READ,WRITE}_ONCE(), using the following
      coccinelle script:
      // Convert trivial ACCESS_ONCE() uses to equivalent READ_ONCE() and
      // WRITE_ONCE()
      // $ make coccicheck COCCI=/home/mark/once.cocci SPFLAGS="--include-headers" MODE=patch
      virtual patch
      @ depends on patch @
  19. 19 Sep, 2017 1 commit
  20. 09 Sep, 2017 1 commit
    • Jérôme Glisse's avatar
      mm/migrate: new migrate mode MIGRATE_SYNC_NO_COPY · 2916ecc0
      Jérôme Glisse authored
      Introduce a new migration mode that allow to offload the copy to a device
      DMA engine.  This changes the workflow of migration and not all
      address_space migratepage callback can support this.
      This is intended to be use by migrate_vma() which itself is use for thing
      like HMM (see include/linux/hmm.h).
      No additional per-filesystem migratepage testing is needed.  I disables
      MIGRATE_SYNC_NO_COPY in all problematic migratepage() callback and i
      added comment in those to explain why (part of this patch).  The commit
      message is unclear it should say that any callback that wish to support
      this new mode need to be aware of the difference in the migration flow
      from other mode.
      Some of these callbacks do extra locking while copying (aio, zsmalloc,
      balloon, ...) and for DMA to be effective you want to copy multiple
      pages in one DMA operations.  But in the problematic case you can not
      easily hold the extra lock accross multiple call to this callback.
      Usual flow is:
      For each page {
       1 - lock page
       2 - call migratepage() callback
       3 - (extra locking in some migratepage() callback)
       4 - migrate page state (freeze refcount, update page cache, buffer
           head, ...)
       5 - copy page
       6 - (unlock any extra lock of migratepage() callback)
       7 - return from migratepage() callback
       8 - unlock page
      The new mode MIGRATE_SYNC_NO_COPY:
       1 - lock multiple pages
      For each page {
       2 - call migratepage() callback
       3 - abort in all problematic migratepage() callback
       4 - migrate page state (freeze refcount, update page cache, buffer
           head, ...)
      } // finished all calls to migratepage() callback
       5 - DMA copy multiple pages
       6 - unlock all the pages
      To support MIGRATE_SYNC_NO_COPY in the problematic case we would need a
      new callback migratepages() (for instance) that deals with multiple
      pages in one transaction.
      Because the problematic cases are not important for current usage I did
      not wanted to complexify this patchset even more for no good reason.
      Link: http://lkml.kernel.org/r/20170817000548.32038-14-jglisse@redhat.com
      Signed-off-by: default avatarJérôme Glisse <jglisse@redhat.com>
      Cc: Aneesh Kumar <aneesh.kumar@linux.vnet.ibm.com>
      Cc: Balbir Singh <bsingharora@gmail.com>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Dan Williams <dan.j.williams@intel.com>
      Cc: David Nellans <dnellans@nvidia.com>
      Cc: Evgeny Baskakov <ebaskakov@nvidia.com>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: John Hubbard <jhubbard@nvidia.com>
      Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
      Cc: Mark Hairgrove <mhairgrove@nvidia.com>
      Cc: Michal Hocko <mhocko@kernel.org>
      Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
      Cc: Sherry Cheung <SCheung@nvidia.com>
      Cc: Subhash Gutti <sgutti@nvidia.com>
      Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
      Cc: Bob Liu <liubo95@huawei.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
  21. 07 Sep, 2017 1 commit