Add PFN_PTE_SHIFT, update_mmu_cache_range() and flush_dcache_folio().
Change the PG_dcache_clean flag from being per-page to per-folio.
Link: https://lkml.kernel.org/r/20230802151406.3735276-12-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Guo Ren <guoren@kernel.org>
Acked-by: Mike Rapoport (IBM) <rppt@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
C-SKY MMU would pre-fetch invalid pte entries, and it could work with
flush_tlb_fix_spurious_fault, but the additional page fault exceptions
would reduce performance. So flushing the entry of the TLB would prevent
the following spurious page faults. Here is the test code:
define DATA_LEN 4096
define COPY_NUM (504*100)
unsigned char src[DATA_LEN*COPY_NUM] = {0};
unsigned char dst[DATA_LEN*COPY_NUM] = {0};
unsigned char func_src[DATA_LEN*COPY_NUM] = {0};
unsigned char func_dst[DATA_LEN*COPY_NUM] = {0};
void main(void)
{
int j;
for (j = 0; j < COPY_NUM; j++)
memcpy(&dst[j*DATA_LEN], &src[j*DATA_LEN], 4);
}
perf stat -e page-faults ./main.elf
The amount of page fault traps would be reduced in half with the patch.
Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
flush_kernel_dcache_page is a rather confusing interface that implements a
subset of flush_dcache_page by not being able to properly handle page
cache mapped pages.
The only callers left are in the exec code as all other previous callers
were incorrect as they could have dealt with page cache pages. Replace
the calls to flush_kernel_dcache_page with calls to flush_dcache_page,
which for all architectures does either exactly the same thing, can
contains one or more of the following:
1) an optimization to defer the cache flush for page cache pages not
mapped into userspace
2) additional flushing for mapped page cache pages if cache aliases
are possible
Link: https://lkml.kernel.org/r/20210712060928.4161649-7-hch@lst.de
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Cc: Alex Shi <alexs@kernel.org>
Cc: Geoff Levand <geoff@infradead.org>
Cc: Greentime Hu <green.hu@gmail.com>
Cc: Guo Ren <guoren@kernel.org>
Cc: Helge Deller <deller@gmx.de>
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Nick Hu <nickhu@andestech.com>
Cc: Paul Cercueil <paul@crapouillou.net>
Cc: Rich Felker <dalias@libc.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: Ulf Hansson <ulf.hansson@linaro.org>
Cc: Vincent Chen <deanbo422@gmail.com>
Cc: Yoshinori Sato <ysato@users.osdn.me>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
page_mapping_file() is only used by some architectures, and then it
is usually only used in one place. Make it a static inline function
so other architectures don't have to carry this dead code.
Link: https://lkml.kernel.org/r/20210317123011.350118-1-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Acked-by: Mike Rapoport <rppt@linux.ibm.com>
Cc: Huang Ying <ying.huang@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
610 has vipt aliasing issue, so we need to finish the cache flush
apis mentioned in cachetlb.rst to avoid data corruption.
Here is the list of modified apis in the patch:
- flush_kernel_dcache_page (new add)
- flush_dcache_mmap_lock (new add)
- flush_dcache_mmap_unlock (new add)
- flush_kernel_vmap_range (new add)
- invalidate_kernel_vmap_range (new add)
- flush_anon_page (new add)
- flush_cache_range (new add)
- flush_cache_vmap (flush all)
- flush_cache_vunmap (flush all)
- flush_cache_mm (only dcache flush)
- flush_icache_page (just nop)
- copy_from_user_page (remove no need flush)
- copy_to_user_page (remove no need flush)
Change to V2:
- Fixup compile error with xa_lock*(&mapping->i_pages)
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Christoph Hellwig <hch@infradead.org>
We use defer cache flush mechanism to improve the performance of
610, but the implementation is wrong. We fix it up now and update
the mechanism:
- Zero page needn't be flushed.
- If page is file mapping & non-touched in user space, defer flush.
- If page is anon mapping or dirty file mapping, flush immediately.
- In update_mmu_cache finish the defer flush by flush_dcache_page().
For 610 we need take care the dcache aliasing issue:
- VIPT cache with 8K-bytes size per way in 4K page granularity.
Signed-off-by: Guo Ren <ren_guo@c-sky.com>
Cc: Arnd Bergmann <arnd@arndb.de>
