+2008-08-31 Andrey Belevantsev <abel@ispras.ru>
+ Dmitry Melnik <dm@ispras.ru>
+ Dmitry Zhurikhin <zhur@ispras.ru>
+ Alexander Monakov <amonakov@ispras.ru>
+ Maxim Kuvyrkov <maxim@codesourcery.com>
+
+ * sel-sched.h, sel-sched-dump.h, sel-sched-ir.h, sel-sched.c,
+ sel-sched-dump.c, sel-sched-ir.c: New files.
+ * Makefile.in (OBJS-common): Add selective scheduling object
+ files.
+ (sel-sched.o, sel-sched-dump.o, sel-sched-ir.o): New entries.
+ (SEL_SCHED_IR_H, SEL_SCHED_DUMP_H): New entries.
+ (sched-vis.o): Add dependency on $(INSN_ATTR_H).
+ * cfghooks.h (get_cfg_hooks, set_cfg_hooks): New prototypes.
+ * cfghooks.c (get_cfg_hooks, set_cfg_hooks): New functions.
+ (make_forwarder_block): Update loop latch if we have redirected
+ the loop latch edge.
+ * cfgloop.c (get_loop_body_in_custom_order): New function.
+ * cfgloop.h (LOOPS_HAVE_FALLTHRU_PREHEADERS): New enum field.
+ (CP_FALLTHRU_PREHEADERS): Likewise.
+ (get_loop_body_in_custom_order): Declare.
+ * cfgloopmanip.c (has_preds_from_loop): New.
+ (create_preheader): Honor CP_FALLTHRU_PREHEADERS.
+ Assert that the preheader edge will be fall thru when it is set.
+ * common.opt (fsel-sched-bookkeeping, fsel-sched-pipelining,
+ fsel-sched-pipelining-outer-loops, fsel-sched-renaming,
+ fsel-sched-substitution, fselective-scheduling): New flags.
+ * cse.c (hash_rtx_cb): New.
+ (hash_rtx): Use it.
+ * dbgcnt.def (sel_sched_cnt, sel_sched_region_cnt,
+ sel_sched_insn_cnt): New counters.
+ * final.c (compute_alignments): Export. Free dominance info after loop_optimizer_finalize.
+ * genattr.c (main): Output maximal_insn_latency prototype.
+ * genautomata.c (output_default_latencies): New. Factor its code from ...
+ (output_internal_insn_latency_func): ... here.
+ (output_internal_maximal_insn_latency_func): New.
+ (output_maximal_insn_latency_func): New.
+ * hard-reg-set.h (UHOST_BITS_PER_WIDE_INT): Define unconditionally.
+ (struct hard_reg_set_iterator): New.
+ (hard_reg_set_iter_init, hard_reg_set_iter_set,
+ hard_reg_set_iter_next): New functions.
+ (EXECUTE_IF_SET_IN_HARD_REG_SET): New macro.
+ * lists.c (remove_free_INSN_LIST_node,
+ remove_free_EXPR_LIST_node): New functions.
+ * loop-init.c (loop_optimizer_init): When LOOPS_HAVE_FALLTHRU_PREHEADERS,
+ set CP_FALLTHRU_PREHEADERS when calling create_preheaders.
+ (loop_optimizer_finalize): Do not verify flow info after reload.
+ * recog.c (validate_replace_rtx_1): New parameter simplify.
+ Default it to true. Update all uses. Factor out simplifying
+ code to ...
+ (simplify_while_replacing): ... this new function.
+ (validate_replace_rtx_part,
+ validate_replace_rtx_part_nosimplify): New.
+ * recog.h (validate_replace_rtx_part,
+ validate_replace_rtx_part_nosimplify): Declare.
+ * rtl.c (rtx_equal_p_cb): New.
+ (rtx_equal_p): Use it.
+ * rtl.h (rtx_equal_p_cb, hash_rtx_cb): Declare.
+ (remove_free_INSN_LIST_NODE, remove_free_EXPR_LIST_node,
+ debug_bb_n_slim, debug_bb_slim, print_rtl_slim): Likewise.
+ * vecprim.h: Add a vector type for unsigned int.
+ * haifa-sched.c: Include vecprim.h and cfgloop.h.
+ (issue_rate, sched_verbose_param, note_list, dfa_state_size,
+ ready_try, cycle_issued_insns, spec_info): Make global.
+ (readyp): Initialize.
+ (dfa_lookahead): New global variable.
+ (old_max_uid, old_last_basic_block): Remove.
+ (h_i_d): Make it a vector.
+ (INSN_TICK, INTER_TICK, QUEUE_INDEX, INSN_COST): Make them work
+ through HID macro.
+ (after_recovery, adding_bb_to_current_region_p):
+ New variables to handle correct insertion of the recovery code.
+ (struct ready_list): Move declaration to sched-int.h.
+ (rgn_n_insns): Removed.
+ (rtx_vec_t): Move to sched-int.h.
+ (find_insn_reg_weight): Remove.
+ (find_insn_reg_weight1): Rename to find_insn_reg_weight.
+ (haifa_init_h_i_d, haifa_finish_h_i_d):
+ New functions to initialize / finalize haifa instruction data.
+ (extend_h_i_d, init_h_i_d): Rewrite.
+ (unlink_other_notes): Move logic to add_to_note_list. Handle
+ selective scheduler.
+ (ready_lastpos, ready_element, ready_sort, reemit_notes,
+ find_fallthru_edge): Make global, remove static prototypes.
+ (max_issue): Make global. Add privileged_n and state parameters. Use
+ them.
+ (extend_global, extend_all): Removed.
+ (init_before_recovery): Add new param. Fix the handling of the case
+ when we insert a recovery code before the EXIT which has a predecessor
+ with a fallthrough edge to it.
+ (create_recovery_block): Make global. Rename to
+ sched_create_recovery_block. Update.
+ (change_pattern): Rename to sched_change_pattern. Make global.
+ (speculate_insn): Rename to sched_speculate_insn. Make global.
+ Split haifa-specific functionality into ...
+ (haifa_change_pattern): New static function.
+ (sched_extend_bb): New static function.
+ (sched_init_bbs): New function.
+ (current_sched_info): Change type to struct haifa_sched_info.
+ (insn_cost): Adjust for selective scheduling.
+ (dep_cost_1): New function. Move logic from ...
+ (dep_cost): ... here.
+ (dep_cost): Use dep_cost_1.
+ (contributes_to_priority_p): Use sched_deps_info instead of
+ current_sched_info.
+ (priority): Adjust to work with selective scheduling. Process the
+ corner case when all dependencies don't contribute to priority.
+ (rank_for_schedule): Use ds_weak instead of dep_weak.
+ (advance_state): New function. Move logic from ...
+ (advance_one_cycle): ... here.
+ (add_to_note_list, concat_note_lists): New functions.
+ (rm_other_notes): Make static. Adjust for selective scheduling.
+ (remove_notes, restore_other_notes): New functions.
+ (move_insn): Add two arguments. Update assert. Don't call
+ reemit_notes.
+ (choose_ready): Remove lookahead variable, use dfa_lookahead.
+ Remove more_issue, max_points. Move the code to initialize
+ max_lookahead_tries to max_issue.
+ (schedule_block): Remove rgn_n_insns1 parameter. Don't allocate
+ ready. Adjust use of move_insn. Call restore_other_notes.
+ (luid): Remove.
+ (sched_init, sched_finish): Move Haifa-specific initialization/
+ finalization to ...
+ (haifa_sched_init, haifa_sched_finish): ... respectively.
+ New functions.
+ (setup_sched_dump): New function.
+ (haifa_init_only_bb): New static function.
+ (haifa_speculate_insn): New static function.
+ (try_ready): Use haifa_* instead of speculate_insn and
+ change_pattern.
+ (extend_ready, extend_all): Remove.
+ (sched_extend_ready_list, sched_finish_ready_list): New functions.
+ (create_check_block_twin, add_to_speculative_block): Use
+ haifa_insns_init instead of extend_global. Update to use new
+ initialization functions. Change parameter. Factor out code from
+ create_check_block_twin to ...
+ (sched_create_recovery_edges) ... this new function.
+ (add_block): Remove.
+ (sched_scan_info): New.
+ (extend_bb): Use sched_scan_info.
+ (init_bb, extend_insn, init_insn, init_insns_in_bb, sched_scan): New
+ static functions for walking through scheduling region.
+ (sched_luids): New vector variable to replace uid_to_luid.
+ (luids_extend_insn): New function.
+ (sched_max_luid): New variable.
+ (luids_init_insn): New function.
+ (sched_init_luids, sched_finish_luids): New functions.
+ (insn_luid): New debug function.
+ (sched_extend_target): New function.
+ (haifa_init_insn): New static function.
+ (sched_init_only_bb): New hook.
+ (sched_split_block): New hook.
+ (sched_split_block_1): New function.
+ (sched_create_empty_bb): New hook.
+ (sched_create_empty_bb_1): New function.
+ (common_sched_info, ready): New global variables.
+ (current_sched_info_var): Remove.
+ (move_block_after_check): Use common_sched_info.
+ (haifa_luid_for_non_insn): New static function.
+ (init_before_recovery): Use haifa_init_only_bb instead of
+ add_block.
+ (increase_insn_priority): New.
+ * modulo-sched.c: (issue_rate): Remove static declaration.
+ (sms_sched_info): Change type to haifa_sched_info.
+ (sms_sched_deps_info, sms_common_sched_info): New variables.
+ (setup_sched_infos): New.
+ (sms_schedule): Initialize them. Call haifa_sched_init/finish.
+ Do not call regstat_free_calls_crossed.
+ (sms_print_insn): Use const_rtx.
+ * params.def (PARAM_MAX_PIPELINE_REGION_BLOCKS,
+ PARAM_MAX_PIPELINE_REGION_INSNS, PARAM_SELSCHED_MAX_LOOKAHEAD,
+ PARAM_SELSCHED_MAX_SCHED_TIMES, PARAM_SELSCHED_INSNS_TO_RENAME,
+ PARAM_SCHED_MEM_TRUE_DEP_COST): New.
+ * sched-deps.c (sched_deps_info): New. Update all relevant uses of
+ current_sched_info to use it.
+ (enum reg_pending_barrier_mode): Move to sched-int.h.
+ (h_d_i_d): New variable. Initialize to NULL.
+ ({true, output, anti, spec, forward}_dependency_cache): Initialize
+ to NULL.
+ (estimate_dep_weak): Remove static declaration.
+ (sched_has_condition_p): New function. Adjust users of
+ sched_get_condition to use it instead.
+ (conditions_mutex_p): Add arguments indicating which conditions are
+ reversed. Use them.
+ (sched_get_condition_with_rev): Rename from sched_get_condition. Add
+ argument to indicate whether returned condition is reversed. Do not
+ generate new rtx when condition should be reversed; indicate it by
+ setting new argument instead.
+ (add_dependence_list_and_free): Add deps parameter.
+ Update all users. Do not free dependence list when
+ deps context is readonly.
+ (add_insn_mem_dependence, flush_pending_lists): Adjust for readonly
+ contexts.
+ (remove_from_dependence_list, remove_from_both_dependence_lists): New.
+ (remove_from_deps): New. Use the above functions.
+ (cur_insn, can_start_lhs_rhs_p): New static variables.
+ (add_or_update_back_dep_1): Initialize present_dep_type.
+ (haifa_start_insn, haifa_finish_insn, haifa_note_reg_set,
+ haifa_note_reg_clobber, haifa_note_reg_use, haifa_note_mem_dep,
+ haifa_note_dep): New functions implementing dependence hooks for
+ the Haifa scheduler.
+ (note_reg_use, note_reg_set, note_reg_clobber, note_mem_dep,
+ note_dep): New functions.
+ (ds_to_dt, extend_deps_reg_info, maybe_extend_reg_info_p): New
+ functions.
+ (init_deps): Initialize last_reg_pending_barrier and deps->readonly.
+ (free_deps): Initialize deps->reg_last.
+ (sched_analyze_reg, sched_analyze_1, sched_analyze_2,
+ sched_analyze_insn): Update to use dependency hooks infrastructure
+ and readonly contexts.
+ (deps_analyze_insn): New function. Move part of logic from ...
+ (sched_analyze): ... here. Also move some logic to ...
+ (deps_start_bb): ... here. New function.
+ (add_forw_dep, delete_forw_dep): Guard use of INSN_DEP_COUNT with
+ sel_sched_p.
+ (sched_deps_init): New function. Move code from ...
+ (init_dependency_caches): ... here. Remove.
+ (init_deps_data_vector): New.
+ (sched_deps_finish): New function. Move code from ...
+ (free_dependency_caches): ... here. Remove.
+ (init_deps_global, finish_deps_global): Adjust for use with
+ selective scheduling.
+ (get_dep_weak): Move logic to ...
+ (get_dep_weak_1): New function.
+ (ds_merge): Move logic to ...
+ (ds_merge_1): New static function.
+ (ds_full_merge, ds_max_merge, ds_get_speculation_types): New functions.
+ (ds_get_max_dep_weak): New function.
+ * sched-ebb.c (sched_n_insns): Rename to sched_rgn_n_insns.
+ (n_insns): Rename to rgn_n_insns.
+ (debug_ebb_dependencies): New function.
+ (init_ready_list): Use it.
+ (begin_schedule_ready): Use sched_init_only_bb.
+ (ebb_print_insn): Indicate when an insn starts a new cycle.
+ (contributes_to_priority, compute_jump_reg_dependencies,
+ add_remove_insn, fix_recovery_cfg): Add ebb_ prefix to function names.
+ (add_block1): Remove to ebb_add_block.
+ (ebb_sched_deps_info, ebb_common_sched_info): New variables.
+ (schedule_ebb): Initialize them. Use remove_notes instead of
+ rm_other_notes. Use haifa_local_init/finish.
+ (schedule_ebbs): Use haifa_sched_init/finish.
+ * sched-int.h: Include vecprim.h, remove rtl.h.
+ (struct ready_list): Delete declaration.
+ (sched_verbose_param, enum sched_pass_id_t,
+ bb_vec_t, insn_vec_t, rtx_vec_t): New.
+ (struct sched_scan_info_def): New structure.
+ (sched_scan_info, sched_scan, sched_init_bbs,
+ sched_init_luids, sched_finish_luids, sched_extend_target,
+ haifa_init_h_i_d, haifa_finish_h_i_d): Declare.
+ (struct common_sched_info_def): New.
+ (common_sched_info, haifa_common_sched_info,
+ sched_emulate_haifa_p): Declare.
+ (sel_sched_p): New.
+ (sched_luids): Declare.
+ (INSN_LUID, LUID_BY_UID, SET_INSN_LUID): Declare.
+ (sched_max_luid, insn_luid): Declare.
+ (note_list, remove_notes, restore_other_notes, bb_note): Declare.
+ (sched_insns_init, sched_insns_finish, xrecalloc, reemit_notes,
+ print_insn, print_pattern, print_value, haifa_classify_insn,
+ sel_find_rgns, sel_mark_hard_insn, dfa_state_size, advance_state,
+ setup_sched_dump, sched_init, sched_finish,
+ sel_insn_is_speculation_check): Export.
+ (struct ready_list): Move from haifa-sched.c.
+ (ready_try, ready, max_issue): Export.
+ (ebb_compute_jump_reg_dependencies, find_fallthru_edge,
+ sched_init_only_bb, sched_split_block, sched_split_block_1,
+ sched_create_empty_bb, sched_create_empty_bb_1,
+ sched_create_recovery_block, sched_create_recovery_edges): Export.
+ (enum reg_pending_barrier_mode): Export.
+ (struct deps): New fields `last_reg_pending_barrier' and `readonly'.
+ (deps_t): New.
+ (struct sched_info): Rename to haifa_sched_info. Use const_rtx for
+ print_insn field. Move add_block and fix_recovery_cfg to
+ common_sched_info_def. Move compute_jump_reg_dependencies, use_cselib ...
+ (struct sched_deps_info_def): ... this new structure.
+ (sched_deps_info): Declare.
+ (struct spec_info_def): Remove weakness_cutoff, add
+ data_weakness_cutoff and control_weakness_cutoff.
+ (spec_info): Declare.
+ (struct _haifa_deps_insn_data): Split from haifa_insn_data. Add
+ dep_count field.
+ (struct haifa_insn_data): Rename to struct _haifa_insn_data.
+ (haifa_insn_data_def, haifa_insn_data_t): New typedefs.
+ (current_sched_info): Change type to struct haifa_sched_info.
+ (haifa_deps_insn_data_def, haifa_deps_insn_data_t): New typedefs.
+ (h_d_i_d): New variable.
+ (HDID): New accessor macro.
+ (h_i_d): Change type to VEC (haifa_insn_data_def, heap) *.
+ (HID): New accessor macro. Rewrite h_i_d accessor macros through HID
+ and HDID.
+ (IS_SPECULATION_CHECK_P): Update for selective scheduler.
+ (enum SCHED_FLAGS): Update for selective scheduler.
+ (enum SPEC_SCHED_FLAGS): New flag SEL_SCHED_SPEC_DONT_CHECK_CONTROL.
+ (init_dependency_caches, free_dependency_caches): Delete declarations.
+ (deps_analyze_insn, remove_from_deps, get_dep_weak_1,
+ estimate_dep_weak, ds_full_merge, ds_max_merge, ds_weak,
+ ds_get_speculation_types, ds_get_max_dep_weak, sched_deps_init,
+ sched_deps_finish, haifa_note_reg_set, haifa_note_reg_use,
+ haifa_note_reg_clobber, maybe_extend_reg_info_p, deps_start_bb,
+ ds_to_dt): Export.
+ (rm_other_notes): Delete declaration.
+ (schedule_block): Remove one argument.
+ (cycle_issued_insns, issue_rate, dfa_lookahead, ready_sort,
+ ready_element, ready_lastpos, sched_extend_ready_list,
+ sched_finish_ready_list, sched_change_pattern, sched_speculate_insn,
+ concat_note_lists): Export.
+ (struct region): Move from sched-rgn.h.
+ (nr_regions, rgn_table, rgn_bb_table, block_to_bb, containing_rgn,
+ RGN_NR_BLOCKS, RGN_BLOCKS, RGN_DONT_CALC_DEPS, RGN_HAS_REAL_EBB,
+ BLOCK_TO_BB, CONTAINING_RGN): Export.
+ (ebb_head, BB_TO_BLOCK, EBB_FIRST_BB, EBB_LAST_BB, INSN_BB): Likewise.
+ (current_nr_blocks, current_blocks, target_bb): Likewise.
+ (dep_cost_1, sched_is_disabled_for_current_region_p, sched_rgn_init,
+ sched_rgn_finish, rgn_setup_region, sched_rgn_compute_dependencies,
+ sched_rgn_local_init, extend_regions,
+ rgn_make_new_region_out_of_new_block, compute_priorities,
+ debug_rgn_dependencies, free_rgn_deps, contributes_to_priority,
+ extend_rgns, deps_join rgn_setup_common_sched_info,
+ rgn_setup_sched_infos, debug_regions, debug_region, dump_region_dot,
+ dump_region_dot_file, haifa_sched_init, haifa_sched_finish): Export.
+ (get_rgn_sched_max_insns_priority, sel_add_to_insn_priority,
+ increase_insn_priority): Likewise.
+ * sched-rgn.c: Include sel-sched.h.
+ (ref_counts): New static variable. Use it ...
+ (INSN_REF_COUNT): ... here. Rewrite and move closer to uses.
+ (FED_BY_SPEC_LOAD, IS_LOAD_INSN): Rewrite to use HID accessor macro.
+ (sched_is_disabled_for_current_region_p): Delete static declaration.
+ (struct region): Move to sched-int.h.
+ (nr_regions, rgn_table, rgn_bb_table, block_to_bb, containing_rgn,
+ ebb_head): Define and initialize.
+ (RGN_NR_BLOCKS, RGN_BLOCKS, RGN_DONT_CALC_DEPS, RGN_HAS_REAL_EBB,
+ BLOCK_TO_BB, CONTAINING_RGN, debug_regions, extend_regions,
+ BB_TO_BLOCK, EBB_FIRST_BB, EBB_LAST_BB): Move to
+ sched-int.h.
+ (find_single_block_region): Add new argument to indicate that EBB
+ regions should be constructed.
+ (debug_live): Delete declaration.
+ (current_nr_blocks, current_blocks, target_bb): Remove static qualifiers.
+ (compute_dom_prob_ps, check_live, update_live, set_spec_fed): Delete
+ declaration.
+ (init_regions): Delete declaration.
+ (debug_region, bb_in_region_p, dump_region_dot_file, dump_region_dot,
+ rgn_estimate_number_of_insns): New.
+ (too_large): Use estimate_number_of_insns.
+ (haifa_find_rgns): New. Move the code from ...
+ (find_rgns): ... here. Call either sel_find_rgns or haifa_find_rgns.
+ (free_trg_info): New.
+ (compute_trg_info): Allocate candidate tables here instead of ...
+ (init_ready_list): ... here.
+ (rgn_print_insn): Use const_rtx.
+ (contributes_to_priority, extend_regions): Delete static declaration.
+ (add_remove_insn, fix_recovery_cfg): Add rgn_ to function names.
+ (add_block1): Rename to rgn_add_block.
+ (debug_rgn_dependencies): Delete static qualifier.
+ (new_ready): Use sched_deps_info. Simplify.
+ (rgn_common_sched_info, rgn_const_sched_deps_info,
+ rgn_const_sel_sched_deps_info, rgn_sched_deps_info, rgn_sched_info): New.
+ (region_sched_info): Rename to rgn_const_sched_info.
+ (deps_join): New, extracted from ...
+ (propagate_deps): ... here.
+ (compute_block_dependences, debug_dependencies): Update for selective
+ scheduling.
+ (free_rgn_deps, compute_priorities): New functions.
+ (sched_rgn_init, sched_rgn_finish, rgn_setup_region,
+ sched_rgn_compute_dependencies): New functions.
+ (schedule_region): Use them.
+ (sched_rgn_local_init, sched_rgn_local_free, sched_rgn_local_finish,
+ rgn_setup_common_sched_info, rgn_setup_sched_infos):
+ New functions.
+ (schedule_insns): Call new functions that were split out.
+ (rgn_make_new_region_out_of_new_block): New.
+ (get_rgn_sched_max_insns_priority): New.
+ (rest_of_handle_sched, rest_of_handle_sched2): Call selective
+ scheduling when appropriate.
+ * sched-vis.c: Include insn-attr.h.
+ (print_value, print_pattern): Make global.
+ (print_rtl_slim, debug_bb_slim, debug_bb_n_slim): New functions.
+ * target-def.h (TARGET_SCHED_ADJUST_COST_2,
+ TARGET_SCHED_ALLOC_SCHED_CONTEXT, TARGET_SCHED_INIT_SCHED_CONTEXT,
+ TARGET_SCHED_SET_SCHED_CONTEXT, TARGET_SCHED_CLEAR_SCHED_CONTEXT,
+ TARGET_SCHED_FREE_SCHED_CONTEXT, TARGET_SCHED_GET_INSN_CHECKED_DS,
+ TARGET_SCHED_GET_INSN_SPEC_DS, TARGET_SCHED_SKIP_RTX_P): New target
+ hooks. Initialize them to 0.
+ (TARGET_SCHED_GEN_CHECK): Rename to TARGET_SCHED_GEN_SPEC_CHECK.
+ * target.h (struct gcc_target): Add them. Rename gen_check field to
+ gen_spec_check.
+ * flags.h (sel_sched_switch_set): Declare.
+ * opts.c (sel_sched_switch_set): New variable.
+ (decode_options): Unset flag_sel_sched_pipelining_outer_loops if
+ pipelining is disabled from command line.
+ (common_handle_option): Record whether selective scheduling is
+ requested from command line.
+ * doc/invoke.texi: Document new flags and parameters.
+ * doc/tm.texi: Document new target hooks.
+ * config/ia64/ia64.c (TARGET_SCHED_GEN_SPEC_CHECK): Define to ia64_gen_check.
+ (dfa_state_size): Do not declare locally.
+ * config/ia64/ia64.opt (msched-ar-data-spec): Default to 0.
+ * config/rs6000/rs6000.c (rs6000_init_sched_context,
+ rs6000_alloc_sched_context, rs6000_set_sched_context,
+ rs6000_free_sched_context): New functions.
+ (struct _rs6000_sched_context): New.
+ (rs6000_sched_reorder2): Do not modify INSN_PRIORITY for selective
+ scheduling.
+ (rs6000_sched_finish): Do not run for selective scheduling.
+
2008-08-31 Jan Hubicka <jh@suse.cz>
* frv.c (frv_rtx_costs): Update forward declaration.
REGS_H = regs.h varray.h $(MACHMODE_H) $(OBSTACK_H) $(BASIC_BLOCK_H) $(FUNCTION_H)
RA_H = ra.h $(REGS_H)
RESOURCE_H = resource.h hard-reg-set.h
-SCHED_INT_H = sched-int.h $(INSN_ATTR_H) $(BASIC_BLOCK_H) $(RTL_H) $(DF_H)
+SCHED_INT_H = sched-int.h $(INSN_ATTR_H) $(BASIC_BLOCK_H) $(RTL_H) $(DF_H) vecprim.h
+SEL_SCHED_IR_H = sel-sched-ir.h $(INSN_ATTR_H) $(BASIC_BLOCK_H) $(RTL_H) \
+ $(GGC_H) $(SCHED_INT_H)
+SEL_SCHED_DUMP_H = sel-sched-dump.h $(SEL_SCHED_IR_H)
INTEGRATE_H = integrate.h $(VARRAY_H)
CFGLAYOUT_H = cfglayout.h $(BASIC_BLOCK_H)
CFGLOOP_H = cfgloop.h $(BASIC_BLOCK_H) $(RTL_H) vecprim.h double-int.h
sched-vis.o \
sdbout.o \
see.o \
+ sel-sched-ir.o \
+ sel-sched-dump.o \
+ sel-sched.o \
simplify-rtx.o \
sparseset.o \
sreal.o \
sched-rgn.o : sched-rgn.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
$(RTL_H) $(SCHED_INT_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) insn-config.h \
$(FUNCTION_H) $(INSN_ATTR_H) $(TOPLEV_H) $(RECOG_H) except.h $(PARAMS_H) \
- $(TM_P_H) $(TARGET_H) $(CFGLAYOUT_H) $(TIMEVAR_H) tree-pass.h $(DBGCNT_H)
+ $(TM_P_H) $(TARGET_H) $(CFGLAYOUT_H) $(TIMEVAR_H) tree-pass.h \
+ $(DBGCNT_H)
sched-ebb.o : sched-ebb.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
$(RTL_H) $(SCHED_INT_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) insn-config.h \
$(FUNCTION_H) $(INSN_ATTR_H) $(TOPLEV_H) $(RECOG_H) except.h $(TM_P_H) \
$(PARAMS_H) $(CFGLAYOUT_H) $(TARGET_H) output.h
sched-vis.o : sched-vis.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
$(RTL_H) $(SCHED_INT_H) hard-reg-set.h $(BASIC_BLOCK_H) $(OBSTACK_H) \
- $(REAL_H) tree-pass.h
+ $(REAL_H) tree-pass.h $(INSN_ATTR_H)
+sel-sched.o : sel-sched.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
+ $(RTL_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) insn-config.h \
+ $(FUNCTION_H) $(INSN_ATTR_H) toplev.h $(RECOG_H) except.h $(PARAMS_H) \
+ $(TM_P_H) $(TARGET_H) $(CFGLAYOUT_H) $(TIMEVAR_H) tree-pass.h \
+ $(SCHED_INT_H) $(GGC_H) $(TREE_H) $(LANGHOOKS_DEF_H) \
+ $(SEL_SCHED_IR_H) $(SEL_SCHED_DUMP_H) sel-sched.h
+sel-sched-dump.o : sel-sched-dump.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
+ $(RTL_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) insn-config.h \
+ $(FUNCTION_H) $(INSN_ATTR_H) toplev.h $(RECOG_H) except.h $(PARAMS_H) \
+ $(TM_P_H) $(TARGET_H) $(CFGLAYOUT_H) $(TIMEVAR_H) tree-pass.h \
+ $(SEL_SCHED_DUMP_H) $(GGC_H) $(TREE_H) $(LANGHOOKS_DEF_H) $(SEL_SCHED_IR_H)
+sel-sched-ir.o : sel-sched-ir.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
+ $(RTL_H) $(REGS_H) hard-reg-set.h $(FLAGS_H) insn-config.h \
+ $(FUNCTION_H) $(INSN_ATTR_H) toplev.h $(RECOG_H) except.h $(PARAMS_H) \
+ $(TM_P_H) $(TARGET_H) $(CFGLAYOUT_H) $(TIMEVAR_H) tree-pass.h \
+ $(SCHED_INT_H) $(GGC_H) $(TREE_H) $(LANGHOOKS_DEF_H) $(SEL_SCHED_IR_H)
final.o : final.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_H) \
$(TREE_H) $(FLAGS_H) intl.h $(REGS_H) $(RECOG_H) conditions.h \
insn-config.h $(INSN_ATTR_H) $(FUNCTION_H) output.h hard-reg-set.h \
cfg_hooks = &gimple_cfg_hooks;
}
+struct cfg_hooks
+get_cfg_hooks (void)
+{
+ return *cfg_hooks;
+}
+
+void
+set_cfg_hooks (struct cfg_hooks new_cfg_hooks)
+{
+ *cfg_hooks = new_cfg_hooks;
+}
+
/* Returns current ir type. */
enum ir_type
/* Redirect back edges we want to keep. */
for (ei = ei_start (dummy->preds); (e = ei_safe_edge (ei)); )
{
+ basic_block e_src;
+
if (redirect_edge_p (e))
{
ei_next (&ei);
if (fallthru->count < 0)
fallthru->count = 0;
+ e_src = e->src;
jump = redirect_edge_and_branch_force (e, bb);
- if (jump != NULL
- && new_bb_cbk != NULL)
- new_bb_cbk (jump);
+ if (jump != NULL)
+ {
+ /* If we redirected the loop latch edge, the JUMP block now acts like
+ the new latch of the loop. */
+ if (current_loops != NULL
+ && dummy->loop_father != NULL
+ && dummy->loop_father->header == dummy
+ && dummy->loop_father->latch == e_src)
+ dummy->loop_father->latch = jump;
+
+ if (new_bb_cbk != NULL)
+ new_bb_cbk (jump);
+ }
}
if (dom_info_available_p (CDI_DOMINATORS))
extern void rtl_register_cfg_hooks (void);
extern void cfg_layout_rtl_register_cfg_hooks (void);
extern void gimple_register_cfg_hooks (void);
+extern struct cfg_hooks get_cfg_hooks (void);
+extern void set_cfg_hooks (struct cfg_hooks);
#endif /* GCC_CFGHOOKS_H */
return tovisit;
}
+/* Gets body of a LOOP sorted via provided BB_COMPARATOR. */
+
+basic_block *
+get_loop_body_in_custom_order (const struct loop *loop,
+ int (*bb_comparator) (const void *, const void *))
+{
+ basic_block *bbs = get_loop_body (loop);
+
+ qsort (bbs, loop->num_nodes, sizeof (basic_block), bb_comparator);
+
+ return bbs;
+}
+
/* Get body of a LOOP in breadth first sort order. */
basic_block *
LOOPS_HAVE_RECORDED_EXITS = 8,
LOOPS_MAY_HAVE_MULTIPLE_LATCHES = 16,
LOOP_CLOSED_SSA = 32,
- LOOPS_NEED_FIXUP = 64
+ LOOPS_NEED_FIXUP = 64,
+ LOOPS_HAVE_FALLTHRU_PREHEADERS = 128
};
#define LOOPS_NORMAL (LOOPS_HAVE_PREHEADERS | LOOPS_HAVE_SIMPLE_LATCHES \
unsigned);
extern basic_block *get_loop_body_in_dom_order (const struct loop *);
extern basic_block *get_loop_body_in_bfs_order (const struct loop *);
+extern basic_block *get_loop_body_in_custom_order (const struct loop *,
+ int (*) (const void *, const void *));
+
extern VEC (edge, heap) *get_loop_exit_edges (const struct loop *);
edge single_exit (const struct loop *);
extern unsigned num_loop_branches (const struct loop *);
enum
{
- CP_SIMPLE_PREHEADERS = 1
+ CP_SIMPLE_PREHEADERS = 1,
+ CP_FALLTHRU_PREHEADERS = 2
};
basic_block create_preheader (struct loop *, int);
return e != mfb_kj_edge;
}
+/* True when a candidate preheader BLOCK has predecessors from LOOP. */
+
+static bool
+has_preds_from_loop (basic_block block, struct loop *loop)
+{
+ edge e;
+ edge_iterator ei;
+
+ FOR_EACH_EDGE (e, ei, block->preds)
+ if (e->src->loop_father == loop)
+ return true;
+ return false;
+}
+
/* Creates a pre-header for a LOOP. Returns newly created block. Unless
CP_SIMPLE_PREHEADERS is set in FLAGS, we only force LOOP to have single
entry; otherwise we also force preheader block to have only one successor.
+ When CP_FALLTHRU_PREHEADERS is set in FLAGS, we force the preheader block
+ to be a fallthru predecessor to the loop header and to have only
+ predecessors from outside of the loop.
The function also updates dominators. */
basic_block
gcc_assert (nentry);
if (nentry == 1)
{
- if (/* We do not allow entry block to be the loop preheader, since we
+ bool need_forwarder_block = false;
+
+ /* We do not allow entry block to be the loop preheader, since we
cannot emit code there. */
- single_entry->src != ENTRY_BLOCK_PTR
- /* If we want simple preheaders, also force the preheader to have
- just a single successor. */
- && !((flags & CP_SIMPLE_PREHEADERS)
- && !single_succ_p (single_entry->src)))
+ if (single_entry->src == ENTRY_BLOCK_PTR)
+ need_forwarder_block = true;
+ else
+ {
+ /* If we want simple preheaders, also force the preheader to have
+ just a single successor. */
+ if ((flags & CP_SIMPLE_PREHEADERS)
+ && !single_succ_p (single_entry->src))
+ need_forwarder_block = true;
+ /* If we want fallthru preheaders, also create forwarder block when
+ preheader ends with a jump or has predecessors from loop. */
+ else if ((flags & CP_FALLTHRU_PREHEADERS)
+ && (JUMP_P (BB_END (single_entry->src))
+ || has_preds_from_loop (single_entry->src, loop)))
+ need_forwarder_block = true;
+ }
+ if (! need_forwarder_block)
return NULL;
}
if (dump_file)
fprintf (dump_file, "Created preheader block for loop %i\n",
loop->num);
+
+ if (flags & CP_FALLTHRU_PREHEADERS)
+ gcc_assert ((single_succ_edge (dummy)->flags & EDGE_FALLTHRU)
+ && !JUMP_P (BB_END (dummy)));
return dummy;
}
Common Report Var(flag_schedule_insns_after_reload) Optimization
Reschedule instructions after register allocation
+; This flag should be on when a target implements non-trivial
+; scheduling hooks, maybe saving some information for its own sake.
+; On IA64, for example, this is used for correct bundling.
+fselective-scheduling
+Common Report Var(flag_selective_scheduling) Optimization
+Schedule instructions using selective scheduling algorithm
+
+fselective-scheduling2
+Common Report Var(flag_selective_scheduling2) Optimization
+Run selective scheduling after reload
+
+fsel-sched-pipelining
+Common Report Var(flag_sel_sched_pipelining) Init(0) Optimization
+Perform software pipelining of inner loops during selective scheduling
+
+fsel-sched-pipelining-outer-loops
+Common Report Var(flag_sel_sched_pipelining_outer_loops) Init(0) Optimization
+Perform software pipelining of outer loops during selective scheduling
+
+fsel-sched-reschedule-pipelined
+Common Report Var(flag_sel_sched_reschedule_pipelined) Init(0) Optimization
+Reschedule pipelined regions without pipelining
+
; sched_stalled_insns means that insns can be moved prematurely from the queue
; of stalled insns into the ready list.
fsched-stalled-insns
#undef TARGET_SCHED_NEEDS_BLOCK_P
#define TARGET_SCHED_NEEDS_BLOCK_P ia64_needs_block_p
-#undef TARGET_SCHED_GEN_CHECK
-#define TARGET_SCHED_GEN_CHECK ia64_gen_check
+#undef TARGET_SCHED_GEN_SPEC_CHECK
+#define TARGET_SCHED_GEN_SPEC_CHECK ia64_gen_check
#undef TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC
#define TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC\
static rtx last_scheduled_insn;
-/* The following variable value is size of the DFA state. */
-
-static size_t dfa_state_size;
-
/* The following variable value is pointer to a DFA state used as
temporary variable. */
Use data speculation before reload
msched-ar-data-spec
-Target Report Var(mflag_sched_ar_data_spec) Init(1)
+Target Report Var(mflag_sched_ar_data_spec) Init(0)
Use data speculation after reload
msched-control-spec
static int rs6000_sched_reorder2 (FILE *, int, rtx *, int *, int);
static int rs6000_use_sched_lookahead (void);
static int rs6000_use_sched_lookahead_guard (rtx);
+static void * rs6000_alloc_sched_context (void);
+static void rs6000_init_sched_context (void *, bool);
+static void rs6000_set_sched_context (void *);
+static void rs6000_free_sched_context (void *);
static tree rs6000_builtin_reciprocal (unsigned int, bool, bool);
static tree rs6000_builtin_mask_for_load (void);
static tree rs6000_builtin_mul_widen_even (tree);
#undef TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD
#define TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD rs6000_use_sched_lookahead_guard
+#undef TARGET_SCHED_ALLOC_SCHED_CONTEXT
+#define TARGET_SCHED_ALLOC_SCHED_CONTEXT rs6000_alloc_sched_context
+#undef TARGET_SCHED_INIT_SCHED_CONTEXT
+#define TARGET_SCHED_INIT_SCHED_CONTEXT rs6000_init_sched_context
+#undef TARGET_SCHED_SET_SCHED_CONTEXT
+#define TARGET_SCHED_SET_SCHED_CONTEXT rs6000_set_sched_context
+#undef TARGET_SCHED_FREE_SCHED_CONTEXT
+#define TARGET_SCHED_FREE_SCHED_CONTEXT rs6000_free_sched_context
+
#undef TARGET_VECTORIZE_BUILTIN_MASK_FOR_LOAD
#define TARGET_VECTORIZE_BUILTIN_MASK_FOR_LOAD rs6000_builtin_mask_for_load
#undef TARGET_VECTORIZE_BUILTIN_MUL_WIDEN_EVEN
for (i=pos; i<*pn_ready-1; i++)
ready[i] = ready[i + 1];
ready[*pn_ready-1] = tmp;
- if INSN_PRIORITY_KNOWN (tmp)
+
+ if (!sel_sched_p () && INSN_PRIORITY_KNOWN (tmp))
INSN_PRIORITY (tmp)++;
break;
}
while (pos >= 0)
{
if (is_load_insn (ready[pos])
- && INSN_PRIORITY_KNOWN (ready[pos]))
+ && !sel_sched_p ()
+ && INSN_PRIORITY_KNOWN (ready[pos]))
{
INSN_PRIORITY (ready[pos])++;
for (i=pos; i<*pn_ready-1; i++)
ready[i] = ready[i + 1];
ready[*pn_ready-1] = tmp;
- if INSN_PRIORITY_KNOWN (tmp)
+
+ if (!sel_sched_p () && INSN_PRIORITY_KNOWN (tmp))
INSN_PRIORITY (tmp)++;
+
first_store_pos = -1;
break;
for (i=first_store_pos; i<*pn_ready-1; i++)
ready[i] = ready[i + 1];
ready[*pn_ready-1] = tmp;
- if INSN_PRIORITY_KNOWN (tmp)
+ if (!sel_sched_p () && INSN_PRIORITY_KNOWN (tmp))
INSN_PRIORITY (tmp)++;
}
}
while (pos >= 0)
{
if (is_store_insn (ready[pos])
- && INSN_PRIORITY_KNOWN (ready[pos]))
+ && !sel_sched_p ()
+ && INSN_PRIORITY_KNOWN (ready[pos]))
{
INSN_PRIORITY (ready[pos])++;
if (group_end)
{
/* If the scheduler had marked group termination at this location
- (between insn and next_indn), and neither insn nor next_insn will
+ (between insn and next_insn), and neither insn nor next_insn will
force group termination, pad the group with nops to force group
termination. */
if (can_issue_more
if (reload_completed && rs6000_sched_groups)
{
+ /* Do not run sched_finish hook when selective scheduling enabled. */
+ if (sel_sched_p ())
+ return;
+
if (rs6000_sched_insert_nops == sched_finish_none)
return;
}
}
}
+
+struct _rs6000_sched_context
+{
+ short cached_can_issue_more;
+ rtx last_scheduled_insn;
+ int load_store_pendulum;
+};
+
+typedef struct _rs6000_sched_context rs6000_sched_context_def;
+typedef rs6000_sched_context_def *rs6000_sched_context_t;
+
+/* Allocate store for new scheduling context. */
+static void *
+rs6000_alloc_sched_context (void)
+{
+ return xmalloc (sizeof (rs6000_sched_context_def));
+}
+
+/* If CLEAN_P is true then initializes _SC with clean data,
+ and from the global context otherwise. */
+static void
+rs6000_init_sched_context (void *_sc, bool clean_p)
+{
+ rs6000_sched_context_t sc = (rs6000_sched_context_t) _sc;
+
+ if (clean_p)
+ {
+ sc->cached_can_issue_more = 0;
+ sc->last_scheduled_insn = NULL_RTX;
+ sc->load_store_pendulum = 0;
+ }
+ else
+ {
+ sc->cached_can_issue_more = cached_can_issue_more;
+ sc->last_scheduled_insn = last_scheduled_insn;
+ sc->load_store_pendulum = load_store_pendulum;
+ }
+}
+
+/* Sets the global scheduling context to the one pointed to by _SC. */
+static void
+rs6000_set_sched_context (void *_sc)
+{
+ rs6000_sched_context_t sc = (rs6000_sched_context_t) _sc;
+
+ gcc_assert (sc != NULL);
+
+ cached_can_issue_more = sc->cached_can_issue_more;
+ last_scheduled_insn = sc->last_scheduled_insn;
+ load_store_pendulum = sc->load_store_pendulum;
+}
+
+/* Free _SC. */
+static void
+rs6000_free_sched_context (void *_sc)
+{
+ gcc_assert (_sc != NULL);
+
+ free (_sc);
+}
+
\f
/* Length in units of the trampoline for entering a nested function. */
static inline unsigned canon_hash (rtx, enum machine_mode);
static inline unsigned safe_hash (rtx, enum machine_mode);
-static unsigned hash_rtx_string (const char *);
+static inline unsigned hash_rtx_string (const char *);
static rtx canon_reg (rtx, rtx);
static enum rtx_code find_comparison_args (enum rtx_code, rtx *, rtx *,
return plus_constant (q->exp, offset);
}
\f
+
/* Hash a string. Just add its bytes up. */
static inline unsigned
hash_rtx_string (const char *ps)
return hash;
}
-/* Hash an rtx. We are careful to make sure the value is never negative.
- Equivalent registers hash identically.
- MODE is used in hashing for CONST_INTs only;
- otherwise the mode of X is used.
-
- Store 1 in DO_NOT_RECORD_P if any subexpression is volatile.
-
- If HASH_ARG_IN_MEMORY_P is not NULL, store 1 in it if X contains
- a MEM rtx which does not have the RTX_UNCHANGING_P bit set.
-
- Note that cse_insn knows that the hash code of a MEM expression
- is just (int) MEM plus the hash code of the address. */
+/* Same as hash_rtx, but call CB on each rtx if it is not NULL.
+ When the callback returns true, we continue with the new rtx. */
unsigned
-hash_rtx (const_rtx x, enum machine_mode mode, int *do_not_record_p,
- int *hash_arg_in_memory_p, bool have_reg_qty)
+hash_rtx_cb (const_rtx x, enum machine_mode mode,
+ int *do_not_record_p, int *hash_arg_in_memory_p,
+ bool have_reg_qty, hash_rtx_callback_function cb)
{
int i, j;
unsigned hash = 0;
enum rtx_code code;
const char *fmt;
+ enum machine_mode newmode;
+ rtx newx;
/* Used to turn recursion into iteration. We can't rely on GCC's
tail-recursion elimination since we need to keep accumulating values
if (x == 0)
return hash;
+ /* Invoke the callback first. */
+ if (cb != NULL
+ && ((*cb) (x, mode, &newx, &newmode)))
+ {
+ hash += hash_rtx_cb (newx, newmode, do_not_record_p,
+ hash_arg_in_memory_p, have_reg_qty, cb);
+ return hash;
+ }
+
code = GET_CODE (x);
switch (code)
{
{
unsigned int regno = REGNO (x);
- if (!reload_completed)
+ if (do_not_record_p && !reload_completed)
{
/* On some machines, we can't record any non-fixed hard register,
because extending its life will cause reload problems. We
for (i = 0; i < units; ++i)
{
elt = CONST_VECTOR_ELT (x, i);
- hash += hash_rtx (elt, GET_MODE (elt), do_not_record_p,
- hash_arg_in_memory_p, have_reg_qty);
+ hash += hash_rtx_cb (elt, GET_MODE (elt),
+ do_not_record_p, hash_arg_in_memory_p,
+ have_reg_qty, cb);
}
return hash;
case MEM:
/* We don't record if marked volatile or if BLKmode since we don't
know the size of the move. */
- if (MEM_VOLATILE_P (x) || GET_MODE (x) == BLKmode)
+ if (do_not_record_p && (MEM_VOLATILE_P (x) || GET_MODE (x) == BLKmode))
{
*do_not_record_p = 1;
return 0;
case CC0:
case CALL:
case UNSPEC_VOLATILE:
- *do_not_record_p = 1;
- return 0;
+ if (do_not_record_p) {
+ *do_not_record_p = 1;
+ return 0;
+ }
+ else
+ return hash;
+ break;
case ASM_OPERANDS:
- if (MEM_VOLATILE_P (x))
+ if (do_not_record_p && MEM_VOLATILE_P (x))
{
*do_not_record_p = 1;
return 0;
{
for (i = 1; i < ASM_OPERANDS_INPUT_LENGTH (x); i++)
{
- hash += (hash_rtx (ASM_OPERANDS_INPUT (x, i),
- GET_MODE (ASM_OPERANDS_INPUT (x, i)),
- do_not_record_p, hash_arg_in_memory_p,
- have_reg_qty)
+ hash += (hash_rtx_cb (ASM_OPERANDS_INPUT (x, i),
+ GET_MODE (ASM_OPERANDS_INPUT (x, i)),
+ do_not_record_p, hash_arg_in_memory_p,
+ have_reg_qty, cb)
+ hash_rtx_string
- (ASM_OPERANDS_INPUT_CONSTRAINT (x, i)));
+ (ASM_OPERANDS_INPUT_CONSTRAINT (x, i)));
}
hash += hash_rtx_string (ASM_OPERANDS_INPUT_CONSTRAINT (x, 0));
x = XEXP (x, i);
goto repeat;
}
-
- hash += hash_rtx (XEXP (x, i), 0, do_not_record_p,
- hash_arg_in_memory_p, have_reg_qty);
+
+ hash += hash_rtx_cb (XEXP (x, i), 0, do_not_record_p,
+ hash_arg_in_memory_p,
+ have_reg_qty, cb);
break;
case 'E':
for (j = 0; j < XVECLEN (x, i); j++)
- hash += hash_rtx (XVECEXP (x, i, j), 0, do_not_record_p,
- hash_arg_in_memory_p, have_reg_qty);
+ hash += hash_rtx_cb (XVECEXP (x, i, j), 0, do_not_record_p,
+ hash_arg_in_memory_p,
+ have_reg_qty, cb);
break;
case 's':
return hash;
}
+/* Hash an rtx. We are careful to make sure the value is never negative.
+ Equivalent registers hash identically.
+ MODE is used in hashing for CONST_INTs only;
+ otherwise the mode of X is used.
+
+ Store 1 in DO_NOT_RECORD_P if any subexpression is volatile.
+
+ If HASH_ARG_IN_MEMORY_P is not NULL, store 1 in it if X contains
+ a MEM rtx which does not have the RTX_UNCHANGING_P bit set.
+
+ Note that cse_insn knows that the hash code of a MEM expression
+ is just (int) MEM plus the hash code of the address. */
+
+unsigned
+hash_rtx (const_rtx x, enum machine_mode mode, int *do_not_record_p,
+ int *hash_arg_in_memory_p, bool have_reg_qty)
+{
+ return hash_rtx_cb (x, mode, do_not_record_p,
+ hash_arg_in_memory_p, have_reg_qty, NULL);
+}
+
/* Hash an rtx X for cse via hash_rtx.
Stores 1 in do_not_record if any subexpression is volatile.
Stores 1 in hash_arg_in_memory if X contains a mem rtx which
DEBUG_COUNTER (sched_func)
DEBUG_COUNTER (sched_insn)
DEBUG_COUNTER (sched_region)
+DEBUG_COUNTER (sel_sched_cnt)
+DEBUG_COUNTER (sel_sched_region_cnt)
+DEBUG_COUNTER (sel_sched_insn_cnt)
DEBUG_COUNTER (sms_sched_loop)
DEBUG_COUNTER (split_for_sched2)
DEBUG_COUNTER (tail_call)
-feliminate-unused-debug-symbols -femit-class-debug-always @gol
-fmem-report -fpre-ipa-mem-report -fpost-ipa-mem-report -fprofile-arcs @gol
-frandom-seed=@var{string} -fsched-verbose=@var{n} @gol
+-fsel-sched-verbose -fsel-sched-dump-cfg -fsel-sched-pipelining-verbose @gol
-ftest-coverage -ftime-report -fvar-tracking @gol
-g -g@var{level} -gcoff -gdwarf-2 @gol
-ggdb -gstabs -gstabs+ -gvms -gxcoff -gxcoff+ @gol
-fsched2-use-traces -fsched-spec-load -fsched-spec-load-dangerous @gol
-fsched-stalled-insns-dep[=@var{n}] -fsched-stalled-insns[=@var{n}] @gol
-fschedule-insns -fschedule-insns2 -fsection-anchors -fsee @gol
+-fselective-scheduling -fselective-scheduling2 @gol
+-fsel-sched-pipelining -fsel-sched-pipelining-outer-loops @gol
-fsignaling-nans -fsingle-precision-constant -fsplit-ivs-in-unroller @gol
-fsplit-wide-types -fstack-protector -fstack-protector-all @gol
-fstrict-aliasing -fstrict-overflow -fthread-jumps -ftracer @gol
was modulo scheduled we may want to prevent the later scheduling passes
from changing its schedule, we use this option to control that.
+@item -fselective-scheduling
+@opindex fselective-scheduling
+Schedule instructions using selective scheduling algorithm. Selective
+scheduling runs instead of the first scheduler pass.
+
+@item -fselective-scheduling2
+@opindex fselective-scheduling2
+Schedule instructions using selective scheduling algorithm. Selective
+scheduling runs instead of the second scheduler pass.
+
+@item -fsel-sched-pipelining
+@opindex fsel-sched-pipelining
+Enable software pipelining of innermost loops during selective scheduling.
+This option has no effect until one of @option{-fselective-scheduling} or
+@option{-fselective-scheduling2} is turned on.
+
+@item -fsel-sched-pipelining-outer-loops
+@opindex fsel-sched-pipelining-outer-loops
+When pipelining loops during selective scheduling, also pipeline outer loops.
+This option has no effect until @option{-fsel-sched-pipelining} is turned on.
+
@item -fcaller-saves
@opindex fcaller-saves
Enable values to be allocated in registers that will be clobbered by
The maximum number of blocks in a region to be considered for
interblock scheduling. The default value is 10.
+@item max-pipeline-region-blocks
+The maximum number of blocks in a region to be considered for
+pipelining in the selective scheduler. The default value is 15.
+
@item max-sched-region-insns
The maximum number of insns in a region to be considered for
interblock scheduling. The default value is 100.
+@item max-pipeline-region-insns
+The maximum number of insns in a region to be considered for
+pipelining in the selective scheduler. The default value is 200.
+
@item min-spec-prob
The minimum probability (in percents) of reaching a source block
for interblock speculative scheduling. The default value is 40.
speculative insn will be scheduled.
The default value is 40.
-@item max-last-value-rtl
+@item sched-mem-true-dep-cost
+Minimal distance (in CPU cycles) between store and load targeting same
+memory locations. The default value is 1.
+
+@item selsched-max-lookahead
+The maximum size of the lookahead window of selective scheduling. It is a
+depth of search for available instructions.
+The default value is 50.
+@item selsched-max-sched-times
+The maximum number of times that an instruction will be scheduled during
+selective scheduling. This is the limit on the number of iterations
+through which the instruction may be pipelined. The default value is 2.
+
+@item selsched-max-insns-to-rename
+The maximum number of best instructions in the ready list that are considered
+for renaming in the selective scheduler. The default value is 2.
+
+@item max-last-value-rtl
The maximum size measured as number of RTLs that can be recorded in an expression
in combiner for a pseudo register as last known value of that register. The default
is 10000.
per instruction data structures.
@end deftypefn
+@deftypefn {Target Hook} void * TARGET_SCHED_ALLOC_SCHED_CONTEXT (void)
+Return a pointer to a store large enough to hold target scheduling context.
+@end deftypefn
+
+@deftypefn {Target Hook} void TARGET_SCHED_INIT_SCHED_CONTEXT (void *@var{tc}, bool @var{clean_p})
+Initialize store pointed to by @var{tc} to hold target scheduling context.
+It @var{clean_p} is true then initialize @var{tc} as if scheduler is at the
+beginning of the block. Overwise, make a copy of the current context in
+@var{tc}.
+@end deftypefn
+
+@deftypefn {Target Hook} void TARGET_SCHED_SET_SCHED_CONTEXT (void *@var{tc})
+Copy target scheduling context pointer to by @var{tc} to the current context.
+@end deftypefn
+
+@deftypefn {Target Hook} void TARGET_SCHED_CLEAR_SCHED_CONTEXT (void *@var{tc})
+Deallocate internal data in target scheduling context pointed to by @var{tc}.
+@end deftypefn
+
+@deftypefn {Target Hook} void TARGET_SCHED_FREE_SCHED_CONTEXT (void *@var{tc})
+Deallocate a store for target scheduling context pointed to by @var{tc}.
+@end deftypefn
+
@deftypefn {Target Hook} int TARGET_SCHED_SPECULATE_INSN (rtx @var{insn}, int @var{request}, rtx *@var{new_pat})
This hook is called by the insn scheduler when @var{insn} has only
speculative dependencies and therefore can be scheduled speculatively.
if (after == last_insn)
last_insn = last;
+
return last;
}
/* Compute branch alignments based on frequency information in the
CFG. */
-static unsigned int
+unsigned int
compute_alignments (void)
{
int log, max_skip, max_log;
}
if (dump_file)
- loop_optimizer_finalize ();
+ {
+ loop_optimizer_finalize ();
+ free_dominance_info (CDI_DOMINATORS);
+ }
return 0;
}
extern unsigned HOST_WIDE_INT g_switch_value;
extern bool g_switch_set;
+/* Same for selective scheduling. */
+extern bool sel_sched_switch_set;
+
/* Values of the -falign-* flags: how much to align labels in code.
0 means `use default', 1 means `don't align'.
For each variable, there is an _log variant which is the power
printf (" Use the function if bypass_p returns nonzero for\n");
printf (" the 1st insn. */\n");
printf ("extern int insn_latency (rtx, rtx);\n\n");
+ printf ("/* Maximal insn latency time possible of all bypasses for this insn.\n");
+ printf (" Use the function if bypass_p returns nonzero for\n");
+ printf (" the 1st insn. */\n");
+ printf ("extern int maximal_insn_latency (rtx);\n\n");
printf ("\n#if AUTOMATON_ALTS\n");
printf ("/* The following function returns number of alternative\n");
printf (" reservations of given insn. It may be used for better\n");
fprintf (output_file, "}\n\n");
}
-/* Output function `internal_insn_latency'. */
+/* Output the array holding default latency values. These are used in
+ insn_latency and maximal_insn_latency function implementations. */
static void
-output_internal_insn_latency_func (void)
+output_default_latencies (void)
{
- decl_t decl;
- struct bypass_decl *bypass;
int i, j, col;
+ decl_t decl;
const char *tabletype = "unsigned char";
/* Find the smallest integer type that can hold all the default
tabletype = "int";
}
- fprintf (output_file, "static int\n%s (int %s ATTRIBUTE_UNUSED,\n\tint %s ATTRIBUTE_UNUSED,\n\trtx %s ATTRIBUTE_UNUSED,\n\trtx %s ATTRIBUTE_UNUSED)\n",
- INTERNAL_INSN_LATENCY_FUNC_NAME, INTERNAL_INSN_CODE_NAME,
- INTERNAL_INSN2_CODE_NAME, INSN_PARAMETER_NAME,
- INSN2_PARAMETER_NAME);
- fprintf (output_file, "{\n");
-
- if (DECL_INSN_RESERV (advance_cycle_insn_decl)->insn_num == 0)
- {
- fputs (" return 0;\n}\n\n", output_file);
- return;
- }
-
fprintf (output_file, " static const %s default_latencies[] =\n {",
tabletype);
}
gcc_assert (j == DECL_INSN_RESERV (advance_cycle_insn_decl)->insn_num);
fputs ("\n };\n", output_file);
+}
+
+/* Output function `internal_insn_latency'. */
+static void
+output_internal_insn_latency_func (void)
+{
+ int i;
+ decl_t decl;
+ struct bypass_decl *bypass;
+
+ fprintf (output_file, "static int\n%s (int %s ATTRIBUTE_UNUSED,\n\tint %s ATTRIBUTE_UNUSED,\n\trtx %s ATTRIBUTE_UNUSED,\n\trtx %s ATTRIBUTE_UNUSED)\n",
+ INTERNAL_INSN_LATENCY_FUNC_NAME, INTERNAL_INSN_CODE_NAME,
+ INTERNAL_INSN2_CODE_NAME, INSN_PARAMETER_NAME,
+ INSN2_PARAMETER_NAME);
+ fprintf (output_file, "{\n");
+
+ if (DECL_INSN_RESERV (advance_cycle_insn_decl)->insn_num == 0)
+ {
+ fputs (" return 0;\n}\n\n", output_file);
+ return;
+ }
fprintf (output_file, " if (%s >= %s || %s >= %s)\n return 0;\n",
INTERNAL_INSN_CODE_NAME, ADVANCE_CYCLE_VALUE_NAME,
INTERNAL_INSN_CODE_NAME);
}
+/* Output function `internal_maximum_insn_latency'. */
+static void
+output_internal_maximal_insn_latency_func (void)
+{
+ decl_t decl;
+ struct bypass_decl *bypass;
+ int i;
+ int max;
+
+ fprintf (output_file, "static int\n%s (int %s ATTRIBUTE_UNUSED,\n\trtx %s ATTRIBUTE_UNUSED)\n",
+ "internal_maximal_insn_latency", INTERNAL_INSN_CODE_NAME,
+ INSN_PARAMETER_NAME);
+ fprintf (output_file, "{\n");
+
+ if (DECL_INSN_RESERV (advance_cycle_insn_decl)->insn_num == 0)
+ {
+ fputs (" return 0;\n}\n\n", output_file);
+ return;
+ }
+
+ fprintf (output_file, " switch (%s)\n {\n", INTERNAL_INSN_CODE_NAME);
+ for (i = 0; i < description->decls_num; i++)
+ if (description->decls[i]->mode == dm_insn_reserv
+ && DECL_INSN_RESERV (description->decls[i])->bypass_list)
+ {
+ decl = description->decls [i];
+ max = DECL_INSN_RESERV (decl)->default_latency;
+ fprintf (output_file,
+ " case %d: {",
+ DECL_INSN_RESERV (decl)->insn_num);
+ for (bypass = DECL_INSN_RESERV (decl)->bypass_list;
+ bypass != NULL;
+ bypass = bypass->next)
+ {
+ if (bypass->latency > max)
+ max = bypass->latency;
+ }
+ fprintf (output_file, " return %d; }\n break;\n", max);
+ }
+
+ fprintf (output_file, " }\n return default_latencies[%s];\n}\n\n",
+ INTERNAL_INSN_CODE_NAME);
+}
+
/* The function outputs PHR interface function `insn_latency'. */
static void
output_insn_latency_func (void)
INSN_PARAMETER_NAME, INSN2_PARAMETER_NAME);
}
+/* The function outputs PHR interface function `maximal_insn_latency'. */
+static void
+output_maximal_insn_latency_func (void)
+{
+ fprintf (output_file, "int\n%s (rtx %s)\n",
+ "maximal_insn_latency", INSN_PARAMETER_NAME);
+ fprintf (output_file, "{\n int %s;\n",
+ INTERNAL_INSN_CODE_NAME);
+ output_internal_insn_code_evaluation (INSN_PARAMETER_NAME,
+ INTERNAL_INSN_CODE_NAME, 0);
+ fprintf (output_file, " return %s (%s, %s);\n}\n\n",
+ "internal_maximal_insn_latency",
+ INTERNAL_INSN_CODE_NAME, INSN_PARAMETER_NAME);
+}
+
/* The function outputs PHR interface function `print_reservation'. */
static void
output_print_reservation_func (void)
output_internal_reset_func ();
output_reset_func ();
output_min_insn_conflict_delay_func ();
+ output_default_latencies ();
output_internal_insn_latency_func ();
output_insn_latency_func ();
+ output_internal_maximal_insn_latency_func ();
+ output_maximal_insn_latency_func ();
output_print_reservation_func ();
/* Output function get_cpu_unit_code. */
fprintf (output_file, "\n#if %s\n\n", CPU_UNITS_QUERY_MACRO_NAME);
#include "target.h"
#include "output.h"
#include "params.h"
+#include "vecprim.h"
#include "dbgcnt.h"
+#include "cfgloop.h"
#ifdef INSN_SCHEDULING
machine cycle. It can be defined in the config/mach/mach.h file,
otherwise we set it to 1. */
-static int issue_rate;
+int issue_rate;
/* sched-verbose controls the amount of debugging output the
scheduler prints. It is controlled by -fsched-verbose=N:
either to stderr, or to the dump listing file (-dRS). */
FILE *sched_dump = 0;
-/* Highest uid before scheduling. */
-static int old_max_uid;
-
/* fix_sched_param() is called from toplev.c upon detection
of the -fsched-verbose=N option. */
warning (0, "fix_sched_param: unknown param: %s", param);
}
-struct haifa_insn_data *h_i_d;
+/* This is a placeholder for the scheduler parameters common
+ to all schedulers. */
+struct common_sched_info_def *common_sched_info;
-#define INSN_TICK(INSN) (h_i_d[INSN_UID (INSN)].tick)
-#define INTER_TICK(INSN) (h_i_d[INSN_UID (INSN)].inter_tick)
+#define INSN_TICK(INSN) (HID (INSN)->tick)
+#define INTER_TICK(INSN) (HID (INSN)->inter_tick)
/* If INSN_TICK of an instruction is equal to INVALID_TICK,
then it should be recalculated from scratch. */
/* List of important notes we must keep around. This is a pointer to the
last element in the list. */
-static rtx note_list;
+rtx note_list;
static struct spec_info_def spec_info_var;
/* Description of the speculative part of the scheduling.
If NULL - no speculation. */
-spec_info_t spec_info;
+spec_info_t spec_info = NULL;
/* True, if recovery block was added during scheduling of current block.
Used to determine, if we need to fix INSN_TICKs. */
/* Array used in {unlink, restore}_bb_notes. */
static rtx *bb_header = 0;
-/* Number of basic_blocks. */
-static int old_last_basic_block;
-
/* Basic block after which recovery blocks will be created. */
static basic_block before_recovery;
+/* Basic block just before the EXIT_BLOCK and after recovery, if we have
+ created it. */
+basic_block after_recovery;
+
+/* FALSE if we add bb to another region, so we don't need to initialize it. */
+bool adding_bb_to_current_region_p = true;
+
/* Queues, etc. */
/* An instruction is ready to be scheduled when all insns preceding it
QUEUE_READY - INSN is in ready list.
N >= 0 - INSN queued for X [where NEXT_Q_AFTER (q_ptr, X) == N] cycles. */
-#define QUEUE_INDEX(INSN) (h_i_d[INSN_UID (INSN)].queue_index)
+#define QUEUE_INDEX(INSN) (HID (INSN)->queue_index)
/* The following variable value refers for all current and future
reservations of the processor units. */
/* The following variable value is size of memory representing all
current and future reservations of the processor units. */
-static size_t dfa_state_size;
+size_t dfa_state_size;
/* The following array is used to find the best insn from ready when
the automaton pipeline interface is used. */
-static char *ready_try;
+char *ready_try = NULL;
-/* Describe the ready list of the scheduler.
- VEC holds space enough for all insns in the current region. VECLEN
- says how many exactly.
- FIRST is the index of the element with the highest priority; i.e. the
- last one in the ready list, since elements are ordered by ascending
- priority.
- N_READY determines how many insns are on the ready list. */
+/* The ready list. */
+struct ready_list ready = {NULL, 0, 0, 0};
-struct ready_list
-{
- rtx *vec;
- int veclen;
- int first;
- int n_ready;
-};
-
-/* The pointer to the ready list. */
-static struct ready_list *readyp;
+/* The pointer to the ready list (to be removed). */
+static struct ready_list *readyp = &ready;
/* Scheduling clock. */
static int clock_var;
-/* Number of instructions in current scheduling region. */
-static int rgn_n_insns;
-
static int may_trap_exp (const_rtx, int);
/* Nonzero iff the address is comprised from at most 1 register. */
/* Returns a class that insn with GET_DEST(insn)=x may belong to,
as found by analyzing insn's expression. */
+\f
+static int haifa_luid_for_non_insn (rtx x);
+
+/* Haifa version of sched_info hooks common to all headers. */
+const struct common_sched_info_def haifa_common_sched_info =
+ {
+ NULL, /* fix_recovery_cfg */
+ NULL, /* add_block */
+ NULL, /* estimate_number_of_insns */
+ haifa_luid_for_non_insn, /* luid_for_non_insn */
+ SCHED_PASS_UNKNOWN /* sched_pass_id */
+ };
+
+const struct sched_scan_info_def *sched_scan_info;
+
+/* Mapping from instruction UID to its Logical UID. */
+VEC (int, heap) *sched_luids = NULL;
+
+/* Next LUID to assign to an instruction. */
+int sched_max_luid = 1;
+
+/* Haifa Instruction Data. */
+VEC (haifa_insn_data_def, heap) *h_i_d = NULL;
+
+void (* sched_init_only_bb) (basic_block, basic_block);
+
+/* Split block function. Different schedulers might use different functions
+ to handle their internal data consistent. */
+basic_block (* sched_split_block) (basic_block, rtx);
+
+/* Create empty basic block after the specified block. */
+basic_block (* sched_create_empty_bb) (basic_block);
+
static int
may_trap_exp (const_rtx x, int is_store)
{
return haifa_classify_rtx (PATTERN (insn));
}
-
-/* A typedef for rtx vector. */
-typedef VEC(rtx, heap) *rtx_vec_t;
-
/* Forward declarations. */
static int priority (rtx);
static void queue_insn (rtx, int);
static int schedule_insn (rtx);
static int find_set_reg_weight (const_rtx);
-static void find_insn_reg_weight (basic_block);
-static void find_insn_reg_weight1 (rtx);
+static void find_insn_reg_weight (const_rtx);
static void adjust_priority (rtx);
static void advance_one_cycle (void);
+static void extend_h_i_d (void);
+
/* Notes handling mechanism:
=========================
unlink_other_notes ()). After scheduling the block, these notes are
inserted at the beginning of the block (in schedule_block()). */
-static rtx unlink_other_notes (rtx, rtx);
-static void reemit_notes (rtx);
-
-static rtx *ready_lastpos (struct ready_list *);
static void ready_add (struct ready_list *, rtx, bool);
-static void ready_sort (struct ready_list *);
static rtx ready_remove_first (struct ready_list *);
static void queue_to_ready (struct ready_list *);
static void debug_ready_list (struct ready_list *);
-static void move_insn (rtx);
-
/* The following functions are used to implement multi-pass scheduling
on the first cycle. */
-static rtx ready_element (struct ready_list *, int);
static rtx ready_remove (struct ready_list *, int);
static void ready_remove_insn (rtx);
-static int max_issue (struct ready_list *, int *, int);
static int choose_ready (struct ready_list *, rtx *);
speculative instructions. */
static void extend_h_i_d (void);
-static void extend_ready (int);
static void init_h_i_d (rtx);
static void generate_recovery_code (rtx);
static void process_insn_forw_deps_be_in_spec (rtx, rtx, ds_t);
static void begin_speculative_block (rtx);
static void add_to_speculative_block (rtx);
-static dw_t dep_weak (ds_t);
-static edge find_fallthru_edge (basic_block);
-static void init_before_recovery (void);
-static basic_block create_recovery_block (void);
+static void init_before_recovery (basic_block *);
static void create_check_block_twin (rtx, bool);
static void fix_recovery_deps (basic_block);
-static void change_pattern (rtx, rtx);
-static int speculate_insn (rtx, ds_t, rtx *);
+static void haifa_change_pattern (rtx, rtx);
static void dump_new_block_header (int, basic_block, rtx, rtx);
static void restore_bb_notes (basic_block);
-static void extend_bb (void);
static void fix_jump_move (rtx);
static void move_block_after_check (rtx);
static void move_succs (VEC(edge,gc) **, basic_block);
#endif /* INSN_SCHEDULING */
\f
/* Point to state used for the current scheduling pass. */
-struct sched_info *current_sched_info;
+struct haifa_sched_info *current_sched_info;
\f
#ifndef INSN_SCHEDULING
void
}
#else
-/* Working copy of frontend's sched_info variable. */
-static struct sched_info current_sched_info_var;
-
/* Pointer to the last instruction scheduled. Used by rank_for_schedule,
so that insns independent of the last scheduled insn will be preferred
over dependent instructions. */
/* Cached cost of the instruction. Use below function to get cost of the
insn. -1 here means that the field is not initialized. */
-#define INSN_COST(INSN) (h_i_d[INSN_UID (INSN)].cost)
+#define INSN_COST(INSN) (HID (INSN)->cost)
/* Compute cost of executing INSN.
This is the number of cycles between instruction issue and
HAIFA_INLINE int
insn_cost (rtx insn)
{
- int cost = INSN_COST (insn);
+ int cost;
+
+ if (sel_sched_p ())
+ {
+ if (recog_memoized (insn) < 0)
+ return 0;
+
+ cost = insn_default_latency (insn);
+ if (cost < 0)
+ cost = 0;
+
+ return cost;
+ }
+
+ cost = INSN_COST (insn);
if (cost < 0)
{
This is the number of cycles between instruction issue and
instruction results. */
int
-dep_cost (dep_t link)
+dep_cost_1 (dep_t link, dw_t dw)
{
rtx used = DEP_CON (link);
int cost;
else if (bypass_p (insn))
cost = insn_latency (insn, used);
}
+
- if (targetm.sched.adjust_cost != NULL)
+ if (targetm.sched.adjust_cost_2)
+ {
+ cost = targetm.sched.adjust_cost_2 (used, (int) dep_type, insn, cost,
+ dw);
+ }
+ else if (targetm.sched.adjust_cost != NULL)
{
/* This variable is used for backward compatibility with the
targets. */
return cost;
}
+/* Compute cost of dependence LINK.
+ This is the number of cycles between instruction issue and
+ instruction results. */
+int
+dep_cost (dep_t link)
+{
+ return dep_cost_1 (link, 0);
+}
+
+/* Use this sel-sched.c friendly function in reorder2 instead of increasing
+ INSN_PRIORITY explicitly. */
+void
+increase_insn_priority (rtx insn, int amount)
+{
+ if (!sel_sched_p ())
+ {
+ /* We're dealing with haifa-sched.c INSN_PRIORITY. */
+ if (INSN_PRIORITY_KNOWN (insn))
+ INSN_PRIORITY (insn) += amount;
+ }
+ else
+ {
+ /* In sel-sched.c INSN_PRIORITY is not kept up to date.
+ Use EXPR_PRIORITY instead. */
+ sel_add_to_insn_priority (insn, amount);
+ }
+}
+
/* Return 'true' if DEP should be included in priority calculations. */
static bool
contributes_to_priority_p (dep_t dep)
their producers will increase, and, thus, the
producers will more likely be scheduled, thus,
resolving the dependence. */
- if ((current_sched_info->flags & DO_SPECULATION)
+ if (sched_deps_info->generate_spec_deps
&& !(spec_info->flags & COUNT_SPEC_IN_CRITICAL_PATH)
&& (DEP_STATUS (dep) & SPECULATIVE))
return false;
if (!INSN_PRIORITY_KNOWN (insn))
{
- int this_priority = 0;
+ int this_priority = -1;
if (sd_lists_empty_p (insn, SD_LIST_FORW))
/* ??? We should set INSN_PRIORITY to insn_cost when and insn has
INSN_FORW_DEPS list of each instruction in the corresponding
recovery block. */
- rec = RECOVERY_BLOCK (insn);
+ /* Selective scheduling does not define RECOVERY_BLOCK macro. */
+ rec = sel_sched_p () ? NULL : RECOVERY_BLOCK (insn);
if (!rec || rec == EXIT_BLOCK_PTR)
{
prev_first = PREV_INSN (insn);
}
while (twin != prev_first);
}
+
+ if (this_priority < 0)
+ {
+ gcc_assert (this_priority == -1);
+
+ this_priority = insn_cost (insn);
+ }
+
INSN_PRIORITY (insn) = this_priority;
INSN_PRIORITY_STATUS (insn) = 1;
}
ds1 = TODO_SPEC (tmp) & SPECULATIVE;
if (ds1)
- dw1 = dep_weak (ds1);
+ dw1 = ds_weak (ds1);
else
dw1 = NO_DEP_WEAK;
ds2 = TODO_SPEC (tmp2) & SPECULATIVE;
if (ds2)
- dw2 = dep_weak (ds2);
+ dw2 = ds_weak (ds2);
else
dw2 = NO_DEP_WEAK;
fprintf (sched_dump, "queued for %d cycles.\n", n_cycles);
}
-
+
QUEUE_INDEX (insn) = next_q;
}
/* Return a pointer to the bottom of the ready list, i.e. the insn
with the lowest priority. */
-HAIFA_INLINE static rtx *
+rtx *
ready_lastpos (struct ready_list *ready)
{
gcc_assert (ready->n_ready >= 1);
insn with the highest priority is 0, and the lowest priority has
N_READY - 1. */
-HAIFA_INLINE static rtx
+rtx
ready_element (struct ready_list *ready, int index)
{
gcc_assert (ready->n_ready && index < ready->n_ready);
/* Sort the ready list READY by ascending priority, using the SCHED_SORT
macro. */
-HAIFA_INLINE static void
+void
ready_sort (struct ready_list *ready)
{
rtx *first = ready_lastpos (ready);
targetm.sched.adjust_priority (prev, INSN_PRIORITY (prev));
}
-/* Advance time on one cycle. */
-HAIFA_INLINE static void
-advance_one_cycle (void)
+/* Advance DFA state STATE on one cycle. */
+void
+advance_state (state_t state)
{
if (targetm.sched.dfa_pre_advance_cycle)
targetm.sched.dfa_pre_advance_cycle ();
if (targetm.sched.dfa_pre_cycle_insn)
- state_transition (curr_state,
+ state_transition (state,
targetm.sched.dfa_pre_cycle_insn ());
- state_transition (curr_state, NULL);
+ state_transition (state, NULL);
if (targetm.sched.dfa_post_cycle_insn)
- state_transition (curr_state,
+ state_transition (state,
targetm.sched.dfa_post_cycle_insn ());
if (targetm.sched.dfa_post_advance_cycle)
targetm.sched.dfa_post_advance_cycle ();
}
+/* Advance time on one cycle. */
+HAIFA_INLINE static void
+advance_one_cycle (void)
+{
+ advance_state (curr_state);
+ if (sched_verbose >= 6)
+ fprintf (sched_dump, "\n;;\tAdvanced a state.\n");
+}
+
/* Clock at which the previous instruction was issued. */
static int last_clock_var;
/* Functions for handling of notes. */
+/* Insert the INSN note at the end of the notes list. */
+static void
+add_to_note_list (rtx insn, rtx *note_list_end_p)
+{
+ PREV_INSN (insn) = *note_list_end_p;
+ if (*note_list_end_p)
+ NEXT_INSN (*note_list_end_p) = insn;
+ *note_list_end_p = insn;
+}
+
+/* Add note list that ends on FROM_END to the end of TO_ENDP. */
+void
+concat_note_lists (rtx from_end, rtx *to_endp)
+{
+ rtx from_start;
+
+ if (from_end == NULL)
+ /* It's easy when have nothing to concat. */
+ return;
+
+ if (*to_endp == NULL)
+ /* It's also easy when destination is empty. */
+ {
+ *to_endp = from_end;
+ return;
+ }
+
+ from_start = from_end;
+ /* A note list should be traversed via PREV_INSN. */
+ while (PREV_INSN (from_start) != NULL)
+ from_start = PREV_INSN (from_start);
+
+ add_to_note_list (from_start, to_endp);
+ *to_endp = from_end;
+}
+
/* Delete notes beginning with INSN and put them in the chain
of notes ended by NOTE_LIST.
Returns the insn following the notes. */
-
static rtx
unlink_other_notes (rtx insn, rtx tail)
{
/* See sched_analyze to see how these are handled. */
if (NOTE_KIND (insn) != NOTE_INSN_EH_REGION_BEG
&& NOTE_KIND (insn) != NOTE_INSN_EH_REGION_END)
- {
- /* Insert the note at the end of the notes list. */
- PREV_INSN (insn) = note_list;
- if (note_list)
- NEXT_INSN (note_list) = insn;
- note_list = insn;
- }
+ add_to_note_list (insn, ¬e_list);
insn = next;
}
+
+ if (insn == tail)
+ {
+ gcc_assert (sel_sched_p ());
+ return prev;
+ }
+
return insn;
}
/* Return the head and tail pointers of ebb starting at BEG and ending
at END. */
-
void
get_ebb_head_tail (basic_block beg, basic_block end, rtx *headp, rtx *tailp)
{
/* Delete notes between HEAD and TAIL and put them in the chain
of notes ended by NOTE_LIST. */
-
-void
+static void
rm_other_notes (rtx head, rtx tail)
{
rtx next_tail;
if (NOTE_NOT_BB_P (insn))
{
prev = insn;
-
insn = unlink_other_notes (insn, next_tail);
- gcc_assert (prev != tail && prev != head && insn != next_tail);
+ gcc_assert ((sel_sched_p ()
+ || prev != tail) && prev != head && insn != next_tail);
+ }
+ }
+}
+
+/* Same as above, but also process REG_SAVE_NOTEs of HEAD. */
+void
+remove_notes (rtx head, rtx tail)
+{
+ /* rm_other_notes only removes notes which are _inside_ the
+ block---that is, it won't remove notes before the first real insn
+ or after the last real insn of the block. So if the first insn
+ has a REG_SAVE_NOTE which would otherwise be emitted before the
+ insn, it is redundant with the note before the start of the
+ block, and so we have to take it out. */
+ if (INSN_P (head))
+ {
+ rtx note;
+
+ for (note = REG_NOTES (head); note; note = XEXP (note, 1))
+ if (REG_NOTE_KIND (note) == REG_SAVE_NOTE)
+ remove_note (head, note);
+ }
+
+ /* Remove remaining note insns from the block, save them in
+ note_list. These notes are restored at the end of
+ schedule_block (). */
+ rm_other_notes (head, tail);
+}
+
+/* Restore-other-notes: NOTE_LIST is the end of a chain of notes
+ previously found among the insns. Insert them just before HEAD. */
+rtx
+restore_other_notes (rtx head, basic_block head_bb)
+{
+ if (note_list != 0)
+ {
+ rtx note_head = note_list;
+
+ if (head)
+ head_bb = BLOCK_FOR_INSN (head);
+ else
+ head = NEXT_INSN (bb_note (head_bb));
+
+ while (PREV_INSN (note_head))
+ {
+ set_block_for_insn (note_head, head_bb);
+ note_head = PREV_INSN (note_head);
}
+ /* In the above cycle we've missed this note. */
+ set_block_for_insn (note_head, head_bb);
+
+ PREV_INSN (note_head) = PREV_INSN (head);
+ NEXT_INSN (PREV_INSN (head)) = note_head;
+ PREV_INSN (head) = note_list;
+ NEXT_INSN (note_list) = head;
+
+ if (BLOCK_FOR_INSN (head) != head_bb)
+ BB_END (head_bb) = note_list;
+
+ head = note_head;
}
+
+ return head;
}
/* Functions for computation of registers live/usage info. */
/* This function looks for a new register being defined.
If the destination register is already used by the source,
a new register is not needed. */
-
static int
find_set_reg_weight (const_rtx x)
{
return 0;
}
-/* Calculate INSN_REG_WEIGHT for all insns of a block. */
-
-static void
-find_insn_reg_weight (basic_block bb)
-{
- rtx insn, next_tail, head, tail;
-
- get_ebb_head_tail (bb, bb, &head, &tail);
- next_tail = NEXT_INSN (tail);
-
- for (insn = head; insn != next_tail; insn = NEXT_INSN (insn))
- find_insn_reg_weight1 (insn);
-}
-
-/* Calculate INSN_REG_WEIGHT for single instruction.
- Separated from find_insn_reg_weight because of need
- to initialize new instruction in generate_recovery_code. */
+/* Calculate INSN_REG_WEIGHT for INSN. */
static void
-find_insn_reg_weight1 (rtx insn)
+find_insn_reg_weight (const_rtx insn)
{
int reg_weight = 0;
rtx x;
NOTEs. The REG_SAVE_NOTE note following first one is contains the
saved value for NOTE_BLOCK_NUMBER which is useful for
NOTE_INSN_EH_REGION_{BEG,END} NOTEs. */
-
-static void
+void
reemit_notes (rtx insn)
{
rtx note, last = insn;
/* Move INSN. Reemit notes if needed. Update CFG, if needed. */
static void
-move_insn (rtx insn)
+move_insn (rtx insn, rtx last, rtx nt)
{
- rtx last = last_scheduled_insn;
-
if (PREV_INSN (insn) != last)
{
basic_block bb;
jump_p = control_flow_insn_p (insn);
gcc_assert (!jump_p
- || ((current_sched_info->flags & SCHED_RGN)
+ || ((common_sched_info->sched_pass_id == SCHED_RGN_PASS)
&& IS_SPECULATION_BRANCHY_CHECK_P (insn))
- || (current_sched_info->flags & SCHED_EBB));
+ || (common_sched_info->sched_pass_id
+ == SCHED_EBB_PASS));
gcc_assert (BLOCK_FOR_INSN (PREV_INSN (insn)) == bb);
if (jump_p)
/* We move the block note along with jump. */
{
- /* NT is needed for assertion below. */
- rtx nt = current_sched_info->next_tail;
+ gcc_assert (nt);
note = NEXT_INSN (insn);
while (NOTE_NOT_BB_P (note) && note != nt)
if (BB_END (bb) == last)
BB_END (bb) = insn;
}
-
- reemit_notes (insn);
SCHED_GROUP_P (insn) = 0;
}
/* The following variable value is number of essential insns issued on
the current cycle. An insn is essential one if it changes the
processors state. */
-static int cycle_issued_insns;
+int cycle_issued_insns;
+
+/* This holds the value of the target dfa_lookahead hook. */
+int dfa_lookahead;
/* The following variable value is maximal number of tries of issuing
insns for the first cycle multipass insn scheduling. We define
of all instructions in READY. The function stops immediately,
if it reached the such a solution, that all instruction can be issued.
INDEX will contain index of the best insn in READY. The following
- function is used only for first cycle multipass scheduling. */
-static int
-max_issue (struct ready_list *ready, int *index, int max_points)
+ function is used only for first cycle multipass scheduling.
+
+ PRIVILEGED_N >= 0
+
+ This function expects recognized insns only. All USEs,
+ CLOBBERs, etc must be filtered elsewhere. */
+int
+max_issue (struct ready_list *ready, int privileged_n, state_t state,
+ int *index)
{
- int n, i, all, n_ready, best, delay, tries_num, points = -1;
+ int n, i, all, n_ready, best, delay, tries_num, points = -1, max_points;
+ int more_issue;
struct choice_entry *top;
rtx insn;
+ n_ready = ready->n_ready;
+ gcc_assert (dfa_lookahead >= 1 && privileged_n >= 0
+ && privileged_n <= n_ready);
+
+ /* Init MAX_LOOKAHEAD_TRIES. */
+ if (cached_first_cycle_multipass_dfa_lookahead != dfa_lookahead)
+ {
+ cached_first_cycle_multipass_dfa_lookahead = dfa_lookahead;
+ max_lookahead_tries = 100;
+ for (i = 0; i < issue_rate; i++)
+ max_lookahead_tries *= dfa_lookahead;
+ }
+
+ /* Init max_points. */
+ max_points = 0;
+ more_issue = issue_rate - cycle_issued_insns;
+ gcc_assert (more_issue >= 0);
+
+ for (i = 0; i < n_ready; i++)
+ if (!ready_try [i])
+ {
+ if (more_issue-- > 0)
+ max_points += ISSUE_POINTS (ready_element (ready, i));
+ else
+ break;
+ }
+
+ /* The number of the issued insns in the best solution. */
best = 0;
- memcpy (choice_stack->state, curr_state, dfa_state_size);
+
top = choice_stack;
- top->rest = cached_first_cycle_multipass_dfa_lookahead;
+
+ /* Set initial state of the search. */
+ memcpy (top->state, state, dfa_state_size);
+ top->rest = dfa_lookahead;
top->n = 0;
- n_ready = ready->n_ready;
+
+ /* Count the number of the insns to search among. */
for (all = i = 0; i < n_ready; i++)
if (!ready_try [i])
all++;
+
+ /* I is the index of the insn to try next. */
i = 0;
tries_num = 0;
for (;;)
{
- if (top->rest == 0 || i >= n_ready)
+ if (/* If we've reached a dead end or searched enough of what we have
+ been asked... */
+ top->rest == 0
+ /* Or have nothing else to try. */
+ || i >= n_ready)
{
+ /* ??? (... || i == n_ready). */
+ gcc_assert (i <= n_ready);
+
if (top == choice_stack)
break;
- if (best < top - choice_stack && ready_try [0])
+
+ if (best < top - choice_stack)
{
- best = top - choice_stack;
- *index = choice_stack [1].index;
- points = top->n;
- if (top->n == max_points || best == all)
- break;
+ if (privileged_n)
+ {
+ n = privileged_n;
+ /* Try to find issued privileged insn. */
+ while (n && !ready_try[--n]);
+ }
+
+ if (/* If all insns are equally good... */
+ privileged_n == 0
+ /* Or a privileged insn will be issued. */
+ || ready_try[n])
+ /* Then we have a solution. */
+ {
+ best = top - choice_stack;
+ /* This is the index of the insn issued first in this
+ solution. */
+ *index = choice_stack [1].index;
+ points = top->n;
+ if (top->n == max_points || best == all)
+ break;
+ }
}
+
+ /* Set ready-list index to point to the last insn
+ ('i++' below will advance it to the next insn). */
i = top->index;
+
+ /* Backtrack. */
ready_try [i] = 0;
top--;
- memcpy (curr_state, top->state, dfa_state_size);
+ memcpy (state, top->state, dfa_state_size);
}
else if (!ready_try [i])
{
if (tries_num > max_lookahead_tries)
break;
insn = ready_element (ready, i);
- delay = state_transition (curr_state, insn);
+ delay = state_transition (state, insn);
if (delay < 0)
{
- if (state_dead_lock_p (curr_state))
+ if (state_dead_lock_p (state))
top->rest = 0;
else
top->rest--;
+
n = top->n;
- if (memcmp (top->state, curr_state, dfa_state_size) != 0)
+ if (memcmp (top->state, state, dfa_state_size) != 0)
n += ISSUE_POINTS (insn);
+
+ /* Advance to the next choice_entry. */
top++;
- top->rest = cached_first_cycle_multipass_dfa_lookahead;
+ /* Initialize it. */
+ top->rest = dfa_lookahead;
top->index = i;
top->n = n;
- memcpy (top->state, curr_state, dfa_state_size);
+ memcpy (top->state, state, dfa_state_size);
+
ready_try [i] = 1;
i = -1;
}
}
+
+ /* Increase ready-list index. */
i++;
}
- while (top != choice_stack)
- {
- ready_try [top->index] = 0;
- top--;
- }
- memcpy (curr_state, choice_stack->state, dfa_state_size);
- if (sched_verbose >= 4)
- fprintf (sched_dump, ";;\t\tChoosed insn : %s; points: %d/%d\n",
- (*current_sched_info->print_insn) (ready_element (ready, *index),
- 0),
- points, max_points);
-
+ /* Restore the original state of the DFA. */
+ memcpy (state, choice_stack->state, dfa_state_size);
+
return best;
}
/* The following function chooses insn from READY and modifies
- *N_READY and READY. The following function is used only for first
+ READY. The following function is used only for first
cycle multipass scheduling.
Return:
-1 if cycle should be advanced,
/* Try to choose the better insn. */
int index = 0, i, n;
rtx insn;
- int more_issue, max_points, try_data = 1, try_control = 1;
+ int try_data = 1, try_control = 1;
+ ds_t ts;
- if (cached_first_cycle_multipass_dfa_lookahead != lookahead)
- {
- cached_first_cycle_multipass_dfa_lookahead = lookahead;
- max_lookahead_tries = 100;
- for (i = 0; i < issue_rate; i++)
- max_lookahead_tries *= lookahead;
- }
insn = ready_element (ready, 0);
if (INSN_CODE (insn) < 0)
{
}
}
- if ((!try_data && (TODO_SPEC (insn) & DATA_SPEC))
- || (!try_control && (TODO_SPEC (insn) & CONTROL_SPEC))
- || (targetm.sched.first_cycle_multipass_dfa_lookahead_guard_spec
- && !targetm.sched.first_cycle_multipass_dfa_lookahead_guard_spec
- (insn)))
+ ts = TODO_SPEC (insn);
+ if ((ts & SPECULATIVE)
+ && (((!try_data && (ts & DATA_SPEC))
+ || (!try_control && (ts & CONTROL_SPEC)))
+ || (targetm.sched.first_cycle_multipass_dfa_lookahead_guard_spec
+ && !targetm.sched
+ .first_cycle_multipass_dfa_lookahead_guard_spec (insn))))
/* Discard speculative instruction that stands first in the ready
list. */
{
return 1;
}
- max_points = ISSUE_POINTS (insn);
- more_issue = issue_rate - cycle_issued_insns - 1;
+ ready_try[0] = 0;
for (i = 1; i < ready->n_ready; i++)
{
insn = ready_element (ready, i);
+
ready_try [i]
- = (INSN_CODE (insn) < 0
- || (!try_data && (TODO_SPEC (insn) & DATA_SPEC))
- || (!try_control && (TODO_SPEC (insn) & CONTROL_SPEC))
- || (targetm.sched.first_cycle_multipass_dfa_lookahead_guard
- && !targetm.sched.first_cycle_multipass_dfa_lookahead_guard
- (insn)));
-
- if (!ready_try [i] && more_issue-- > 0)
- max_points += ISSUE_POINTS (insn);
+ = ((!try_data && (TODO_SPEC (insn) & DATA_SPEC))
+ || (!try_control && (TODO_SPEC (insn) & CONTROL_SPEC)));
}
- if (max_issue (ready, &index, max_points) == 0)
+ /* Let the target filter the search space. */
+ for (i = 1; i < ready->n_ready; i++)
+ if (!ready_try[i])
+ {
+ insn = ready_element (ready, i);
+
+ gcc_assert (INSN_CODE (insn) >= 0
+ || recog_memoized (insn) < 0);
+
+ ready_try [i]
+ = (/* INSN_CODE check can be omitted here as it is also done later
+ in max_issue (). */
+ INSN_CODE (insn) < 0
+ || (targetm.sched.first_cycle_multipass_dfa_lookahead_guard
+ && !targetm.sched.first_cycle_multipass_dfa_lookahead_guard
+ (insn)));
+ }
+
+ if (max_issue (ready, 1, curr_state, &index) == 0)
{
+ if (sched_verbose >= 4)
+ fprintf (sched_dump, ";;\t\tChosen none\n");
*insn_ptr = ready_remove_first (ready);
return 0;
}
else
{
+ if (sched_verbose >= 4)
+ fprintf (sched_dump, ";;\t\tChosen insn : %s\n",
+ (*current_sched_info->print_insn)
+ (ready_element (ready, index), 0));
+
*insn_ptr = ready_remove (ready, index);
return 0;
}
region. */
void
-schedule_block (basic_block *target_bb, int rgn_n_insns1)
+schedule_block (basic_block *target_bb)
{
- struct ready_list ready;
int i, first_cycle_insn_p;
int can_issue_more;
state_t temp_state = NULL; /* It is used for multipass scheduling. */
state_reset (curr_state);
- /* Allocate the ready list. */
- readyp = &ready;
- ready.vec = NULL;
- ready_try = NULL;
- choice_stack = NULL;
-
- rgn_n_insns = -1;
- extend_ready (rgn_n_insns1 + 1);
-
+ /* Clear the ready list. */
ready.first = ready.veclen - 1;
ready.n_ready = 0;
(*current_sched_info->begin_schedule_ready) (insn,
last_scheduled_insn);
- move_insn (insn);
+ move_insn (insn, last_scheduled_insn, current_sched_info->next_tail);
+ reemit_notes (insn);
last_scheduled_insn = insn;
if (memcmp (curr_state, temp_state, dfa_state_size) != 0)
}
}
+ if (sched_verbose)
+ fprintf (sched_dump, ";; total time = %d\n", clock_var);
+
if (!current_sched_info->queue_must_finish_empty
|| haifa_recovery_bb_recently_added_p)
{
if (targetm.sched.md_finish)
{
targetm.sched.md_finish (sched_dump, sched_verbose);
-
/* Target might have added some instructions to the scheduled block
in its md_finish () hook. These new insns don't have any data
initialized and to identify them we extend h_i_d so that they'll
- get zero luids.*/
- extend_h_i_d ();
+ get zero luids. */
+ sched_init_luids (NULL, NULL, NULL, NULL);
}
+ if (sched_verbose)
+ fprintf (sched_dump, ";; new head = %d\n;; new tail = %d\n\n",
+ INSN_UID (head), INSN_UID (tail));
+
/* Update head/tail boundaries. */
head = NEXT_INSN (prev_head);
tail = last_scheduled_insn;
- /* Restore-other-notes: NOTE_LIST is the end of a chain of notes
- previously found among the insns. Insert them at the beginning
- of the insns. */
- if (note_list != 0)
- {
- basic_block head_bb = BLOCK_FOR_INSN (head);
- rtx note_head = note_list;
-
- while (PREV_INSN (note_head))
- {
- set_block_for_insn (note_head, head_bb);
- note_head = PREV_INSN (note_head);
- }
- /* In the above cycle we've missed this note: */
- set_block_for_insn (note_head, head_bb);
-
- PREV_INSN (note_head) = PREV_INSN (head);
- NEXT_INSN (PREV_INSN (head)) = note_head;
- PREV_INSN (head) = note_list;
- NEXT_INSN (note_list) = head;
- head = note_head;
- }
-
- /* Debugging. */
- if (sched_verbose)
- {
- fprintf (sched_dump, ";; total time = %d\n;; new head = %d\n",
- clock_var, INSN_UID (head));
- fprintf (sched_dump, ";; new tail = %d\n\n",
- INSN_UID (tail));
- }
+ head = restore_other_notes (head, NULL);
current_sched_info->head = head;
current_sched_info->tail = tail;
-
- free (ready.vec);
-
- free (ready_try);
- for (i = 0; i <= rgn_n_insns; i++)
- free (choice_stack [i].state);
- free (choice_stack);
}
\f
/* Set_priorities: compute priority of each insn in the block. */
return n_insn;
}
-/* Next LUID to assign to an instruction. */
-static int luid;
+/* Set dump and sched_verbose for the desired debugging output. If no
+ dump-file was specified, but -fsched-verbose=N (any N), print to stderr.
+ For -fsched-verbose=N, N>=10, print everything to stderr. */
+void
+setup_sched_dump (void)
+{
+ sched_verbose = sched_verbose_param;
+ if (sched_verbose_param == 0 && dump_file)
+ sched_verbose = 1;
+ sched_dump = ((sched_verbose_param >= 10 || !dump_file)
+ ? stderr : dump_file);
+}
-/* Initialize some global state for the scheduler. */
+/* Initialize some global state for the scheduler. This function works
+ with the common data shared between all the schedulers. It is called
+ from the scheduler specific initialization routine. */
void
sched_init (void)
{
- basic_block b;
- rtx insn;
- int i;
-
- /* Switch to working copy of sched_info. */
- memcpy (¤t_sched_info_var, current_sched_info,
- sizeof (current_sched_info_var));
- current_sched_info = ¤t_sched_info_var;
-
/* Disable speculative loads in their presence if cc0 defined. */
#ifdef HAVE_cc0
flag_schedule_speculative_load = 0;
#endif
- /* Set dump and sched_verbose for the desired debugging output. If no
- dump-file was specified, but -fsched-verbose=N (any N), print to stderr.
- For -fsched-verbose=N, N>=10, print everything to stderr. */
- sched_verbose = sched_verbose_param;
- if (sched_verbose_param == 0 && dump_file)
- sched_verbose = 1;
- sched_dump = ((sched_verbose_param >= 10 || !dump_file)
- ? stderr : dump_file);
-
/* Initialize SPEC_INFO. */
if (targetm.sched.set_sched_flags)
{
spec_info = &spec_info_var;
targetm.sched.set_sched_flags (spec_info);
- if (current_sched_info->flags & DO_SPECULATION)
- spec_info->weakness_cutoff =
- (PARAM_VALUE (PARAM_SCHED_SPEC_PROB_CUTOFF) * MAX_DEP_WEAK) / 100;
+
+ if (spec_info->mask != 0)
+ {
+ spec_info->data_weakness_cutoff =
+ (PARAM_VALUE (PARAM_SCHED_SPEC_PROB_CUTOFF) * MAX_DEP_WEAK) / 100;
+ spec_info->control_weakness_cutoff =
+ (PARAM_VALUE (PARAM_SCHED_SPEC_PROB_CUTOFF)
+ * REG_BR_PROB_BASE) / 100;
+ }
else
/* So we won't read anything accidentally. */
- spec_info = 0;
+ spec_info = NULL;
+
}
else
/* So we won't read anything accidentally. */
cached_first_cycle_multipass_dfa_lookahead = 0;
}
- old_max_uid = 0;
- h_i_d = 0;
- extend_h_i_d ();
-
- for (i = 0; i < old_max_uid; i++)
- {
- h_i_d[i].cost = -1;
- h_i_d[i].todo_spec = HARD_DEP;
- h_i_d[i].queue_index = QUEUE_NOWHERE;
- h_i_d[i].tick = INVALID_TICK;
- h_i_d[i].inter_tick = INVALID_TICK;
- }
+ if (targetm.sched.first_cycle_multipass_dfa_lookahead)
+ dfa_lookahead = targetm.sched.first_cycle_multipass_dfa_lookahead ();
+ else
+ dfa_lookahead = 0;
if (targetm.sched.init_dfa_pre_cycle_insn)
targetm.sched.init_dfa_pre_cycle_insn ();
dfa_start ();
dfa_state_size = state_size ();
- curr_state = xmalloc (dfa_state_size);
- h_i_d[0].luid = 0;
- luid = 1;
- FOR_EACH_BB (b)
- for (insn = BB_HEAD (b); ; insn = NEXT_INSN (insn))
- {
- INSN_LUID (insn) = luid;
+ init_alias_analysis ();
- /* Increment the next luid, unless this is a note. We don't
- really need separate IDs for notes and we don't want to
- schedule differently depending on whether or not there are
- line-number notes, i.e., depending on whether or not we're
- generating debugging information. */
- if (!NOTE_P (insn))
- ++luid;
+ df_set_flags (DF_LR_RUN_DCE);
+ df_note_add_problem ();
- if (insn == BB_END (b))
- break;
- }
+ /* More problems needed for interloop dep calculation in SMS. */
+ if (common_sched_info->sched_pass_id == SCHED_SMS_PASS)
+ {
+ df_rd_add_problem ();
+ df_chain_add_problem (DF_DU_CHAIN + DF_UD_CHAIN);
+ }
- init_dependency_caches (luid);
+ df_analyze ();
+
+ /* Do not run DCE after reload, as this can kill nops inserted
+ by bundling. */
+ if (reload_completed)
+ df_clear_flags (DF_LR_RUN_DCE);
- init_alias_analysis ();
+ regstat_compute_calls_crossed ();
- old_last_basic_block = 0;
- extend_bb ();
+ if (targetm.sched.md_init_global)
+ targetm.sched.md_init_global (sched_dump, sched_verbose,
+ get_max_uid () + 1);
- /* Compute INSN_REG_WEIGHT for all blocks. We must do this before
- removing death notes. */
- FOR_EACH_BB_REVERSE (b)
- find_insn_reg_weight (b);
+ curr_state = xmalloc (dfa_state_size);
+}
- if (targetm.sched.md_init_global)
- targetm.sched.md_init_global (sched_dump, sched_verbose, old_max_uid);
+static void haifa_init_only_bb (basic_block, basic_block);
- nr_begin_data = nr_begin_control = nr_be_in_data = nr_be_in_control = 0;
- before_recovery = 0;
+/* Initialize data structures specific to the Haifa scheduler. */
+void
+haifa_sched_init (void)
+{
+ setup_sched_dump ();
+ sched_init ();
+
+ if (spec_info != NULL)
+ {
+ sched_deps_info->use_deps_list = 1;
+ sched_deps_info->generate_spec_deps = 1;
+ }
+
+ /* Initialize luids, dependency caches, target and h_i_d for the
+ whole function. */
+ {
+ bb_vec_t bbs = VEC_alloc (basic_block, heap, n_basic_blocks);
+ basic_block bb;
+ sched_init_bbs ();
+
+ FOR_EACH_BB (bb)
+ VEC_quick_push (basic_block, bbs, bb);
+ sched_init_luids (bbs, NULL, NULL, NULL);
+ sched_deps_init (true);
+ sched_extend_target ();
+ haifa_init_h_i_d (bbs, NULL, NULL, NULL);
+
+ VEC_free (basic_block, heap, bbs);
+ }
+
+ sched_init_only_bb = haifa_init_only_bb;
+ sched_split_block = sched_split_block_1;
+ sched_create_empty_bb = sched_create_empty_bb_1;
haifa_recovery_bb_ever_added_p = false;
#ifdef ENABLE_CHECKING
- /* This is used preferably for finding bugs in check_cfg () itself. */
+ /* This is used preferably for finding bugs in check_cfg () itself.
+ We must call sched_bbs_init () before check_cfg () because check_cfg ()
+ assumes that the last insn in the last bb has a non-null successor. */
check_cfg (0, 0);
#endif
-}
-/* Free global data used during insn scheduling. */
+ nr_begin_data = nr_begin_control = nr_be_in_data = nr_be_in_control = 0;
+ before_recovery = 0;
+ after_recovery = 0;
+}
+/* Finish work with the data specific to the Haifa scheduler. */
void
-sched_finish (void)
+haifa_sched_finish (void)
{
- free (h_i_d);
- free (curr_state);
- dfa_finish ();
- free_dependency_caches ();
- end_alias_analysis ();
+ sched_create_empty_bb = NULL;
+ sched_split_block = NULL;
+ sched_init_only_bb = NULL;
- if (targetm.sched.md_finish_global)
- targetm.sched.md_finish_global (sched_dump, sched_verbose);
-
if (spec_info && spec_info->dump)
{
char c = reload_completed ? 'a' : 'b';
c, nr_be_in_control);
}
+ /* Finalize h_i_d, dependency caches, and luids for the whole
+ function. Target will be finalized in md_global_finish (). */
+ sched_deps_finish ();
+ sched_finish_luids ();
+ current_sched_info = NULL;
+ sched_finish ();
+}
+
+/* Free global data used during insn scheduling. This function works with
+ the common data shared between the schedulers. */
+
+void
+sched_finish (void)
+{
+ haifa_finish_h_i_d ();
+ free (curr_state);
+
+ if (targetm.sched.md_finish_global)
+ targetm.sched.md_finish_global (sched_dump, sched_verbose);
+
+ end_alias_analysis ();
+
+ regstat_free_calls_crossed ();
+
+ dfa_finish ();
+
#ifdef ENABLE_CHECKING
/* After reload ia64 backend clobbers CFG, so can't check anything. */
if (!reload_completed)
check_cfg (0, 0);
#endif
-
- current_sched_info = NULL;
}
/* Fix INSN_TICKs of the instructions in the current block as well as
}
bitmap_clear (&processed);
}
+
+static int haifa_speculate_insn (rtx, ds_t, rtx *);
/* Check if NEXT is ready to be added to the ready or queue list.
If "yes", add it to the proper list.
*ts = ds_merge (*ts, ds);
}
- if (dep_weak (*ts) < spec_info->weakness_cutoff)
+ if (ds_weak (*ts) < spec_info->data_weakness_cutoff)
/* Too few points. */
*ts = (*ts & ~SPECULATIVE) | HARD_DEP;
}
gcc_assert ((*ts & SPECULATIVE) && !(*ts & ~SPECULATIVE));
- res = speculate_insn (next, *ts, &new_pat);
+ res = haifa_speculate_insn (next, *ts, &new_pat);
switch (res)
{
save it. */
ORIG_PAT (next) = PATTERN (next);
- change_pattern (next, new_pat);
+ haifa_change_pattern (next, new_pat);
break;
default:
ORIG_PAT field. Except one case - speculation checks have ORIG_PAT
pat too, so skip them. */
{
- change_pattern (next, ORIG_PAT (next));
+ haifa_change_pattern (next, ORIG_PAT (next));
ORIG_PAT (next) = 0;
}
}
}
-/* Extend H_I_D data. */
-static void
-extend_h_i_d (void)
-{
- /* We use LUID 0 for the fake insn (UID 0) which holds dependencies for
- pseudos which do not cross calls. */
- int new_max_uid = get_max_uid () + 1;
-
- h_i_d = (struct haifa_insn_data *)
- xrecalloc (h_i_d, new_max_uid, old_max_uid, sizeof (*h_i_d));
- old_max_uid = new_max_uid;
-
- if (targetm.sched.h_i_d_extended)
- targetm.sched.h_i_d_extended ();
-}
+static int sched_ready_n_insns = -1;
-/* Extend READY, READY_TRY and CHOICE_STACK arrays.
- N_NEW_INSNS is the number of additional elements to allocate. */
-static void
-extend_ready (int n_new_insns)
+/* Initialize per region data structures. */
+void
+sched_extend_ready_list (int new_sched_ready_n_insns)
{
int i;
- readyp->veclen = rgn_n_insns + n_new_insns + 1 + issue_rate;
- readyp->vec = XRESIZEVEC (rtx, readyp->vec, readyp->veclen);
-
- ready_try = (char *) xrecalloc (ready_try, rgn_n_insns + n_new_insns + 1,
- rgn_n_insns + 1, sizeof (char));
+ if (sched_ready_n_insns == -1)
+ /* At the first call we need to initialize one more choice_stack
+ entry. */
+ {
+ i = 0;
+ sched_ready_n_insns = 0;
+ }
+ else
+ i = sched_ready_n_insns + 1;
- rgn_n_insns += n_new_insns;
+ ready.veclen = new_sched_ready_n_insns + issue_rate;
+ ready.vec = XRESIZEVEC (rtx, ready.vec, ready.veclen);
- choice_stack = XRESIZEVEC (struct choice_entry, choice_stack,
- rgn_n_insns + 1);
+ gcc_assert (new_sched_ready_n_insns >= sched_ready_n_insns);
- for (i = rgn_n_insns; n_new_insns--; i--)
- choice_stack[i].state = xmalloc (dfa_state_size);
-}
+ ready_try = (char *) xrecalloc (ready_try, new_sched_ready_n_insns,
+ sched_ready_n_insns, sizeof (*ready_try));
-/* Extend global-scope scheduler data structures
- (those, that live within one call to schedule_insns)
- to include information about just emitted INSN. */
-static void
-extend_global_data (rtx insn)
-{
- gcc_assert (INSN_P (insn));
+ /* We allocate +1 element to save initial state in the choice_stack[0]
+ entry. */
+ choice_stack = XRESIZEVEC (struct choice_entry, choice_stack,
+ new_sched_ready_n_insns + 1);
- /* Init h_i_d. */
- extend_h_i_d ();
- init_h_i_d (insn);
+ for (; i <= new_sched_ready_n_insns; i++)
+ choice_stack[i].state = xmalloc (dfa_state_size);
- /* Extend dependency caches by one element. */
- extend_dependency_caches (1, false);
+ sched_ready_n_insns = new_sched_ready_n_insns;
}
-/* Extend global- and region-scope scheduler data structures
- (those, that live within one call to schedule_region)
- to include information about just emitted INSN. */
-static void
-extend_region_data (rtx insn)
+/* Free per region data structures. */
+void
+sched_finish_ready_list (void)
{
- extend_global_data (insn);
+ int i;
- /* Init dependency data. */
- sd_init_insn (insn);
-}
+ free (ready.vec);
+ ready.vec = NULL;
+ ready.veclen = 0;
-/* Extend global-, region- and block-scope scheduler data structures
- (those, that live within one call to schedule_block)
- to include information about just emitted INSN. */
-static void
-extend_block_data (rtx insn)
-{
- extend_region_data (insn);
+ free (ready_try);
+ ready_try = NULL;
- /* These structures have block scope. */
- extend_ready (1);
+ for (i = 0; i <= sched_ready_n_insns; i++)
+ free (choice_stack [i].state);
+ free (choice_stack);
+ choice_stack = NULL;
- (*current_sched_info->add_remove_insn) (insn, 0);
+ sched_ready_n_insns = -1;
}
-/* Initialize h_i_d entry of the new INSN with default values.
- Values, that are not explicitly initialized here, hold zero. */
-static void
-init_h_i_d (rtx insn)
+static int
+haifa_luid_for_non_insn (rtx x)
{
- INSN_LUID (insn) = luid++;
- INSN_COST (insn) = -1;
- TODO_SPEC (insn) = HARD_DEP;
- QUEUE_INDEX (insn) = QUEUE_NOWHERE;
- INSN_TICK (insn) = INVALID_TICK;
- INTER_TICK (insn) = INVALID_TICK;
- find_insn_reg_weight1 (insn);
+ gcc_assert (NOTE_P (x) || LABEL_P (x));
+
+ return 0;
}
/* Generates recovery code for INSN. */
it can be removed from the ready (or queue) list only
due to backend decision. Hence we can't let the
probability of the speculative dep to decrease. */
- dep_weak (ds) <= dep_weak (fs))
+ ds_weak (ds) <= ds_weak (fs))
{
ds_t new_ds;
TODO_SPEC (insn) &= ~BEGIN_SPEC;
}
+static void haifa_init_insn (rtx);
+
/* Generates recovery code for BE_IN speculative INSN. */
static void
add_to_speculative_block (rtx insn)
rec = BLOCK_FOR_INSN (check);
twin = emit_insn_before (copy_insn (PATTERN (insn)), BB_END (rec));
- extend_region_data (twin);
+ haifa_init_insn (twin);
sd_copy_back_deps (twin, insn, true);
return p;
}
-/* Return the probability of speculation success for the speculation
- status DS. */
-static dw_t
-dep_weak (ds_t ds)
-{
- ds_t res = 1, dt;
- int n = 0;
-
- dt = FIRST_SPEC_TYPE;
- do
- {
- if (ds & dt)
- {
- res *= (ds_t) get_dep_weak (ds, dt);
- n++;
- }
-
- if (dt == LAST_SPEC_TYPE)
- break;
- dt <<= SPEC_TYPE_SHIFT;
- }
- while (1);
-
- gcc_assert (n);
- while (--n)
- res /= MAX_DEP_WEAK;
-
- if (res < MIN_DEP_WEAK)
- res = MIN_DEP_WEAK;
-
- gcc_assert (res <= MAX_DEP_WEAK);
-
- return (dw_t) res;
-}
-
/* Helper function.
Find fallthru edge from PRED. */
-static edge
+edge
find_fallthru_edge (basic_block pred)
{
edge e;
return NULL;
}
+/* Extend per basic block data structures. */
+static void
+sched_extend_bb (void)
+{
+ rtx insn;
+
+ /* The following is done to keep current_sched_info->next_tail non null. */
+ insn = BB_END (EXIT_BLOCK_PTR->prev_bb);
+ if (NEXT_INSN (insn) == 0
+ || (!NOTE_P (insn)
+ && !LABEL_P (insn)
+ /* Don't emit a NOTE if it would end up before a BARRIER. */
+ && !BARRIER_P (NEXT_INSN (insn))))
+ {
+ rtx note = emit_note_after (NOTE_INSN_DELETED, insn);
+ /* Make insn appear outside BB. */
+ set_block_for_insn (note, NULL);
+ BB_END (EXIT_BLOCK_PTR->prev_bb) = insn;
+ }
+}
+
+/* Init per basic block data structures. */
+void
+sched_init_bbs (void)
+{
+ sched_extend_bb ();
+}
+
/* Initialize BEFORE_RECOVERY variable. */
static void
-init_before_recovery (void)
+init_before_recovery (basic_block *before_recovery_ptr)
{
basic_block last;
edge e;
basic_block single, empty;
rtx x, label;
- single = create_empty_bb (last);
- empty = create_empty_bb (single);
+ /* If the fallthrough edge to exit we've found is from the block we've
+ created before, don't do anything more. */
+ if (last == after_recovery)
+ return;
- single->count = last->count;
+ adding_bb_to_current_region_p = false;
+
+ single = sched_create_empty_bb (last);
+ empty = sched_create_empty_bb (single);
+
+ /* Add new blocks to the root loop. */
+ if (current_loops != NULL)
+ {
+ add_bb_to_loop (single, VEC_index (loop_p, current_loops->larray, 0));
+ add_bb_to_loop (empty, VEC_index (loop_p, current_loops->larray, 0));
+ }
+
+ single->count = last->count;
empty->count = last->count;
single->frequency = last->frequency;
empty->frequency = last->frequency;
x = emit_jump_insn_after (gen_jump (label), BB_END (single));
JUMP_LABEL (x) = label;
LABEL_NUSES (label)++;
- extend_global_data (x);
+ haifa_init_insn (x);
emit_barrier_after (x);
- add_block (empty, 0);
- add_block (single, 0);
+ sched_init_only_bb (empty, NULL);
+ sched_init_only_bb (single, NULL);
+ sched_extend_bb ();
+ adding_bb_to_current_region_p = true;
before_recovery = single;
+ after_recovery = empty;
+
+ if (before_recovery_ptr)
+ *before_recovery_ptr = before_recovery;
if (sched_verbose >= 2 && spec_info->dump)
fprintf (spec_info->dump,
}
/* Returns new recovery block. */
-static basic_block
-create_recovery_block (void)
+basic_block
+sched_create_recovery_block (basic_block *before_recovery_ptr)
{
rtx label;
rtx barrier;
haifa_recovery_bb_recently_added_p = true;
haifa_recovery_bb_ever_added_p = true;
- if (!before_recovery)
- init_before_recovery ();
+ init_before_recovery (before_recovery_ptr);
barrier = get_last_bb_insn (before_recovery);
gcc_assert (BARRIER_P (barrier));
rec = create_basic_block (label, label, before_recovery);
- /* Recovery block always end with an unconditional jump. */
+ /* A recovery block always ends with an unconditional jump. */
emit_barrier_after (BB_END (rec));
if (BB_PARTITION (before_recovery) != BB_UNPARTITIONED)
fprintf (spec_info->dump, ";;\t\tGenerated recovery block rec%d\n",
rec->index);
- before_recovery = rec;
-
return rec;
}
+/* Create edges: FIRST_BB -> REC; FIRST_BB -> SECOND_BB; REC -> SECOND_BB
+ and emit necessary jumps. */
+void
+sched_create_recovery_edges (basic_block first_bb, basic_block rec,
+ basic_block second_bb)
+{
+ rtx label;
+ rtx jump;
+ edge e;
+ int edge_flags;
+
+ /* This is fixing of incoming edge. */
+ /* ??? Which other flags should be specified? */
+ if (BB_PARTITION (first_bb) != BB_PARTITION (rec))
+ /* Partition type is the same, if it is "unpartitioned". */
+ edge_flags = EDGE_CROSSING;
+ else
+ edge_flags = 0;
+
+ e = make_edge (first_bb, rec, edge_flags);
+ label = block_label (second_bb);
+ jump = emit_jump_insn_after (gen_jump (label), BB_END (rec));
+ JUMP_LABEL (jump) = label;
+ LABEL_NUSES (label)++;
+
+ if (BB_PARTITION (second_bb) != BB_PARTITION (rec))
+ /* Partition type is the same, if it is "unpartitioned". */
+ {
+ /* Rewritten from cfgrtl.c. */
+ if (flag_reorder_blocks_and_partition
+ && targetm.have_named_sections)
+ /* We don't need the same note for the check because
+ any_condjump_p (check) == true. */
+ {
+ REG_NOTES (jump) = gen_rtx_EXPR_LIST (REG_CROSSING_JUMP,
+ NULL_RTX,
+ REG_NOTES (jump));
+ }
+ edge_flags = EDGE_CROSSING;
+ }
+ else
+ edge_flags = 0;
+
+ make_single_succ_edge (rec, second_bb, edge_flags);
+}
+
/* This function creates recovery code for INSN. If MUTATE_P is nonzero,
INSN is a simple check, that should be converted to branchy one. */
static void
sd_iterator_def sd_it;
dep_t dep;
dep_def _new_dep, *new_dep = &_new_dep;
+ ds_t todo_spec;
- gcc_assert (ORIG_PAT (insn)
- && (!mutate_p
- || (IS_SPECULATION_SIMPLE_CHECK_P (insn)
- && !(TODO_SPEC (insn) & SPECULATIVE))));
+ gcc_assert (ORIG_PAT (insn) != NULL_RTX);
+
+ if (!mutate_p)
+ todo_spec = TODO_SPEC (insn);
+ else
+ {
+ gcc_assert (IS_SPECULATION_SIMPLE_CHECK_P (insn)
+ && (TODO_SPEC (insn) & SPECULATIVE) == 0);
+
+ todo_spec = CHECK_SPEC (insn);
+ }
+
+ todo_spec &= SPECULATIVE;
/* Create recovery block. */
if (mutate_p || targetm.sched.needs_block_p (insn))
{
- rec = create_recovery_block ();
+ rec = sched_create_recovery_block (NULL);
label = BB_HEAD (rec);
}
else
{
rec = EXIT_BLOCK_PTR;
- label = 0;
+ label = NULL_RTX;
}
/* Emit CHECK. */
- check = targetm.sched.gen_check (insn, label, mutate_p);
+ check = targetm.sched.gen_spec_check (insn, label, mutate_p);
if (rec != EXIT_BLOCK_PTR)
{
check = emit_insn_before (check, insn);
/* Extend data structures. */
- extend_block_data (check);
+ haifa_init_insn (check);
+
+ /* CHECK is being added to current region. Extend ready list. */
+ gcc_assert (sched_ready_n_insns != -1);
+ sched_extend_ready_list (sched_ready_n_insns + 1);
+
+ if (current_sched_info->add_remove_insn)
+ current_sched_info->add_remove_insn (insn, 0);
+
RECOVERY_BLOCK (check) = rec;
if (sched_verbose && spec_info->dump)
}
twin = emit_insn_after (ORIG_PAT (insn), BB_END (rec));
- extend_region_data (twin);
+ haifa_init_insn (twin);
if (sched_verbose && spec_info->dump)
/* INSN_BB (insn) isn't determined for twin insns yet.
{
basic_block first_bb, second_bb;
rtx jump;
- edge e;
- int edge_flags;
first_bb = BLOCK_FOR_INSN (check);
- e = split_block (first_bb, check);
- /* split_block emits note if *check == BB_END. Probably it
- is better to rip that note off. */
- gcc_assert (e->src == first_bb);
- second_bb = e->dest;
-
- /* This is fixing of incoming edge. */
- /* ??? Which other flags should be specified? */
- if (BB_PARTITION (first_bb) != BB_PARTITION (rec))
- /* Partition type is the same, if it is "unpartitioned". */
- edge_flags = EDGE_CROSSING;
- else
- edge_flags = 0;
-
- e = make_edge (first_bb, rec, edge_flags);
+ second_bb = sched_split_block (first_bb, check);
- add_block (second_bb, first_bb);
-
- gcc_assert (NOTE_INSN_BASIC_BLOCK_P (BB_HEAD (second_bb)));
- label = block_label (second_bb);
- jump = emit_jump_insn_after (gen_jump (label), BB_END (rec));
- JUMP_LABEL (jump) = label;
- LABEL_NUSES (label)++;
- extend_region_data (jump);
+ sched_create_recovery_edges (first_bb, rec, second_bb);
- if (BB_PARTITION (second_bb) != BB_PARTITION (rec))
- /* Partition type is the same, if it is "unpartitioned". */
- {
- /* Rewritten from cfgrtl.c. */
- if (flag_reorder_blocks_and_partition
- && targetm.have_named_sections
- /*&& !any_condjump_p (jump)*/)
- /* any_condjump_p (jump) == false.
- We don't need the same note for the check because
- any_condjump_p (check) == true. */
- add_reg_note (jump, REG_CROSSING_JUMP, NULL_RTX);
- edge_flags = EDGE_CROSSING;
- }
- else
- edge_flags = 0;
-
- make_single_succ_edge (rec, second_bb, edge_flags);
-
- add_block (rec, EXIT_BLOCK_PTR);
+ sched_init_only_bb (second_bb, first_bb);
+ sched_init_only_bb (rec, EXIT_BLOCK_PTR);
+
+ jump = BB_END (rec);
+ haifa_init_insn (jump);
}
/* Move backward dependences from INSN to CHECK and
add_jump_dependencies (insn, jump);
}
-/* Changes pattern of the INSN to NEW_PAT. */
-static void
-change_pattern (rtx insn, rtx new_pat)
+/* Change pattern of INSN to NEW_PAT. */
+void
+sched_change_pattern (rtx insn, rtx new_pat)
{
int t;
t = validate_change (insn, &PATTERN (insn), new_pat, 0);
gcc_assert (t);
+ dfa_clear_single_insn_cache (insn);
+}
+
+/* Change pattern of INSN to NEW_PAT. Invalidate cached haifa
+ instruction data. */
+static void
+haifa_change_pattern (rtx insn, rtx new_pat)
+{
+ sched_change_pattern (insn, new_pat);
+
/* Invalidate INSN_COST, so it'll be recalculated. */
INSN_COST (insn) = -1;
/* Invalidate INSN_TICK, so it'll be recalculated. */
INSN_TICK (insn) = INVALID_TICK;
- dfa_clear_single_insn_cache (insn);
}
/* -1 - can't speculate,
0 - for speculation with REQUEST mode it is OK to use
current instruction pattern,
1 - need to change pattern for *NEW_PAT to be speculative. */
-static int
-speculate_insn (rtx insn, ds_t request, rtx *new_pat)
+int
+sched_speculate_insn (rtx insn, ds_t request, rtx *new_pat)
{
gcc_assert (current_sched_info->flags & DO_SPECULATION
&& (request & SPECULATIVE)
&& !(request & BEGIN_SPEC))
return 0;
- return targetm.sched.speculate_insn (insn, request & BEGIN_SPEC, new_pat);
+ return targetm.sched.speculate_insn (insn, request, new_pat);
+}
+
+static int
+haifa_speculate_insn (rtx insn, ds_t request, rtx *new_pat)
+{
+ gcc_assert (sched_deps_info->generate_spec_deps
+ && !IS_SPECULATION_CHECK_P (insn));
+
+ if (HAS_INTERNAL_DEP (insn)
+ || SCHED_GROUP_P (insn))
+ return -1;
+
+ return sched_speculate_insn (insn, request, new_pat);
}
/* Print some information about block BB, which starts with HEAD and
bb_header = 0;
}
-/* Extend per basic block data structures of the scheduler.
- If BB is NULL, initialize structures for the whole CFG.
- Otherwise, initialize them for the just created BB. */
-static void
-extend_bb (void)
-{
- rtx insn;
-
- old_last_basic_block = last_basic_block;
-
- /* The following is done to keep current_sched_info->next_tail non null. */
-
- insn = BB_END (EXIT_BLOCK_PTR->prev_bb);
- if (NEXT_INSN (insn) == 0
- || (!NOTE_P (insn)
- && !LABEL_P (insn)
- /* Don't emit a NOTE if it would end up before a BARRIER. */
- && !BARRIER_P (NEXT_INSN (insn))))
- {
- rtx note = emit_note_after (NOTE_INSN_DELETED, insn);
- /* Make insn appear outside BB. */
- set_block_for_insn (note, NULL);
- BB_END (EXIT_BLOCK_PTR->prev_bb) = insn;
- }
-}
-
-/* Add a basic block BB to extended basic block EBB.
- If EBB is EXIT_BLOCK_PTR, then BB is recovery block.
- If EBB is NULL, then BB should be a new region. */
-void
-add_block (basic_block bb, basic_block ebb)
-{
- gcc_assert (current_sched_info->flags & NEW_BBS);
-
- extend_bb ();
-
- if (current_sched_info->add_block)
- /* This changes only data structures of the front-end. */
- current_sched_info->add_block (bb, ebb);
-}
-
/* Helper function.
Fix CFG after both in- and inter-block movement of
control_flow_insn_p JUMP. */
jump_bb = BLOCK_FOR_INSN (jump);
jump_bb_next = jump_bb->next_bb;
- gcc_assert (current_sched_info->flags & SCHED_EBB
+ gcc_assert (common_sched_info->sched_pass_id == SCHED_EBB_PASS
|| IS_SPECULATION_BRANCHY_CHECK_P (jump));
if (!NOTE_INSN_BASIC_BLOCK_P (BB_END (jump_bb_next)))
df_mark_solutions_dirty ();
- if (current_sched_info->fix_recovery_cfg)
- current_sched_info->fix_recovery_cfg
- (bb->index, jump_bb->index, jump_bb_next->index);
+ common_sched_info->fix_recovery_cfg
+ (bb->index, jump_bb->index, jump_bb_next->index);
}
/* Helper function for move_block_after_check.
gcc_assert (bb == 0);
}
+
#endif /* ENABLE_CHECKING */
+const struct sched_scan_info_def *sched_scan_info;
+
+/* Extend per basic block data structures. */
+static void
+extend_bb (void)
+{
+ if (sched_scan_info->extend_bb)
+ sched_scan_info->extend_bb ();
+}
+
+/* Init data for BB. */
+static void
+init_bb (basic_block bb)
+{
+ if (sched_scan_info->init_bb)
+ sched_scan_info->init_bb (bb);
+}
+
+/* Extend per insn data structures. */
+static void
+extend_insn (void)
+{
+ if (sched_scan_info->extend_insn)
+ sched_scan_info->extend_insn ();
+}
+
+/* Init data structures for INSN. */
+static void
+init_insn (rtx insn)
+{
+ if (sched_scan_info->init_insn)
+ sched_scan_info->init_insn (insn);
+}
+
+/* Init all insns in BB. */
+static void
+init_insns_in_bb (basic_block bb)
+{
+ rtx insn;
+
+ FOR_BB_INSNS (bb, insn)
+ init_insn (insn);
+}
+
+/* A driver function to add a set of basic blocks (BBS),
+ a single basic block (BB), a set of insns (INSNS) or a single insn (INSN)
+ to the scheduling region. */
+void
+sched_scan (const struct sched_scan_info_def *ssi,
+ bb_vec_t bbs, basic_block bb, insn_vec_t insns, rtx insn)
+{
+ sched_scan_info = ssi;
+
+ if (bbs != NULL || bb != NULL)
+ {
+ extend_bb ();
+
+ if (bbs != NULL)
+ {
+ unsigned i;
+ basic_block x;
+
+ for (i = 0; VEC_iterate (basic_block, bbs, i, x); i++)
+ init_bb (x);
+ }
+
+ if (bb != NULL)
+ init_bb (bb);
+ }
+
+ extend_insn ();
+
+ if (bbs != NULL)
+ {
+ unsigned i;
+ basic_block x;
+
+ for (i = 0; VEC_iterate (basic_block, bbs, i, x); i++)
+ init_insns_in_bb (x);
+ }
+
+ if (bb != NULL)
+ init_insns_in_bb (bb);
+
+ if (insns != NULL)
+ {
+ unsigned i;
+ rtx x;
+
+ for (i = 0; VEC_iterate (rtx, insns, i, x); i++)
+ init_insn (x);
+ }
+
+ if (insn != NULL)
+ init_insn (insn);
+}
+
+
+/* Extend data structures for logical insn UID. */
+static void
+luids_extend_insn (void)
+{
+ int new_luids_max_uid = get_max_uid () + 1;
+
+ VEC_safe_grow_cleared (int, heap, sched_luids, new_luids_max_uid);
+}
+
+/* Initialize LUID for INSN. */
+static void
+luids_init_insn (rtx insn)
+{
+ int i = INSN_P (insn) ? 1 : common_sched_info->luid_for_non_insn (insn);
+ int luid;
+
+ if (i >= 0)
+ {
+ luid = sched_max_luid;
+ sched_max_luid += i;
+ }
+ else
+ luid = -1;
+
+ SET_INSN_LUID (insn, luid);
+}
+
+/* Initialize luids for BBS, BB, INSNS and INSN.
+ The hook common_sched_info->luid_for_non_insn () is used to determine
+ if notes, labels, etc. need luids. */
+void
+sched_init_luids (bb_vec_t bbs, basic_block bb, insn_vec_t insns, rtx insn)
+{
+ const struct sched_scan_info_def ssi =
+ {
+ NULL, /* extend_bb */
+ NULL, /* init_bb */
+ luids_extend_insn, /* extend_insn */
+ luids_init_insn /* init_insn */
+ };
+
+ sched_scan (&ssi, bbs, bb, insns, insn);
+}
+
+/* Free LUIDs. */
+void
+sched_finish_luids (void)
+{
+ VEC_free (int, heap, sched_luids);
+ sched_max_luid = 1;
+}
+
+/* Return logical uid of INSN. Helpful while debugging. */
+int
+insn_luid (rtx insn)
+{
+ return INSN_LUID (insn);
+}
+
+/* Extend per insn data in the target. */
+void
+sched_extend_target (void)
+{
+ if (targetm.sched.h_i_d_extended)
+ targetm.sched.h_i_d_extended ();
+}
+
+/* Extend global scheduler structures (those, that live across calls to
+ schedule_block) to include information about just emitted INSN. */
+static void
+extend_h_i_d (void)
+{
+ int reserve = (get_max_uid () + 1
+ - VEC_length (haifa_insn_data_def, h_i_d));
+ if (reserve > 0
+ && ! VEC_space (haifa_insn_data_def, h_i_d, reserve))
+ {
+ VEC_safe_grow_cleared (haifa_insn_data_def, heap, h_i_d,
+ 3 * get_max_uid () / 2);
+ sched_extend_target ();
+ }
+}
+
+/* Initialize h_i_d entry of the INSN with default values.
+ Values, that are not explicitly initialized here, hold zero. */
+static void
+init_h_i_d (rtx insn)
+{
+ if (INSN_LUID (insn) > 0)
+ {
+ INSN_COST (insn) = -1;
+ find_insn_reg_weight (insn);
+ QUEUE_INDEX (insn) = QUEUE_NOWHERE;
+ INSN_TICK (insn) = INVALID_TICK;
+ INTER_TICK (insn) = INVALID_TICK;
+ TODO_SPEC (insn) = HARD_DEP;
+ }
+}
+
+/* Initialize haifa_insn_data for BBS, BB, INSNS and INSN. */
+void
+haifa_init_h_i_d (bb_vec_t bbs, basic_block bb, insn_vec_t insns, rtx insn)
+{
+ const struct sched_scan_info_def ssi =
+ {
+ NULL, /* extend_bb */
+ NULL, /* init_bb */
+ extend_h_i_d, /* extend_insn */
+ init_h_i_d /* init_insn */
+ };
+
+ sched_scan (&ssi, bbs, bb, insns, insn);
+}
+
+/* Finalize haifa_insn_data. */
+void
+haifa_finish_h_i_d (void)
+{
+ VEC_free (haifa_insn_data_def, heap, h_i_d);
+}
+
+/* Init data for the new insn INSN. */
+static void
+haifa_init_insn (rtx insn)
+{
+ gcc_assert (insn != NULL);
+
+ sched_init_luids (NULL, NULL, NULL, insn);
+ sched_extend_target ();
+ sched_deps_init (false);
+ haifa_init_h_i_d (NULL, NULL, NULL, insn);
+
+ if (adding_bb_to_current_region_p)
+ {
+ sd_init_insn (insn);
+
+ /* Extend dependency caches by one element. */
+ extend_dependency_caches (1, false);
+ }
+}
+
+/* Init data for the new basic block BB which comes after AFTER. */
+static void
+haifa_init_only_bb (basic_block bb, basic_block after)
+{
+ gcc_assert (bb != NULL);
+
+ sched_init_bbs ();
+
+ if (common_sched_info->add_block)
+ /* This changes only data structures of the front-end. */
+ common_sched_info->add_block (bb, after);
+}
+
+/* A generic version of sched_split_block (). */
+basic_block
+sched_split_block_1 (basic_block first_bb, rtx after)
+{
+ edge e;
+
+ e = split_block (first_bb, after);
+ gcc_assert (e->src == first_bb);
+
+ /* sched_split_block emits note if *check == BB_END. Probably it
+ is better to rip that note off. */
+
+ return e->dest;
+}
+
+/* A generic version of sched_create_empty_bb (). */
+basic_block
+sched_create_empty_bb_1 (basic_block after)
+{
+ return create_empty_bb (after);
+}
+
#endif /* INSN_SCHEDULING */
hard_reg_set_intersect_p (X, Y), which returns true if X and Y intersect.
hard_reg_set_empty_p (X), which returns true if X is empty. */
+#define UHOST_BITS_PER_WIDE_INT ((unsigned) HOST_BITS_PER_WIDEST_FAST_INT)
+
#ifdef HARD_REG_SET
#define SET_HARD_REG_BIT(SET, BIT) \
#else
-#define UHOST_BITS_PER_WIDE_INT ((unsigned) HOST_BITS_PER_WIDEST_FAST_INT)
-
#define SET_HARD_REG_BIT(SET, BIT) \
((SET)[(BIT) / UHOST_BITS_PER_WIDE_INT] \
|= HARD_CONST (1) << ((BIT) % UHOST_BITS_PER_WIDE_INT))
#endif
#endif
+/* Iterator for hard register sets. */
+
+typedef struct
+{
+ /* Pointer to the current element. */
+ HARD_REG_ELT_TYPE *pelt;
+
+ /* The length of the set. */
+ unsigned short length;
+
+ /* Word within the current element. */
+ unsigned short word_no;
+
+ /* Contents of the actually processed word. When finding next bit
+ it is shifted right, so that the actual bit is always the least
+ significant bit of ACTUAL. */
+ HARD_REG_ELT_TYPE bits;
+} hard_reg_set_iterator;
+
+#define HARD_REG_ELT_BITS UHOST_BITS_PER_WIDE_INT
+
+/* The implementation of the iterator functions is fully analogous to
+ the bitmap iterators. */
+static inline void
+hard_reg_set_iter_init (hard_reg_set_iterator *iter, HARD_REG_SET set,
+ unsigned min, unsigned *regno)
+{
+#ifdef HARD_REG_SET_LONGS
+ iter->pelt = set;
+ iter->length = HARD_REG_SET_LONGS;
+#else
+ iter->pelt = &set;
+ iter->length = 1;
+#endif
+ iter->word_no = min / HARD_REG_ELT_BITS;
+ if (iter->word_no < iter->length)
+ {
+ iter->bits = iter->pelt[iter->word_no];
+ iter->bits >>= min % HARD_REG_ELT_BITS;
+
+ /* This is required for correct search of the next bit. */
+ min += !iter->bits;
+ }
+ *regno = min;
+}
+
+static inline bool
+hard_reg_set_iter_set (hard_reg_set_iterator *iter, unsigned *regno)
+{
+ while (1)
+ {
+ /* Return false when we're advanced past the end of the set. */
+ if (iter->word_no >= iter->length)
+ return false;
+
+ if (iter->bits)
+ {
+ /* Find the correct bit and return it. */
+ while (!(iter->bits & 1))
+ {
+ iter->bits >>= 1;
+ *regno += 1;
+ }
+ return (*regno < FIRST_PSEUDO_REGISTER);
+ }
+
+ /* Round to the beginning of the next word. */
+ *regno = (*regno + HARD_REG_ELT_BITS - 1);
+ *regno -= *regno % HARD_REG_ELT_BITS;
+
+ /* Find the next non-zero word. */
+ while (++iter->word_no < iter->length)
+ {
+ iter->bits = iter->pelt[iter->word_no];
+ if (iter->bits)
+ break;
+ *regno += HARD_REG_ELT_BITS;
+ }
+ }
+}
+
+static inline void
+hard_reg_set_iter_next (hard_reg_set_iterator *iter, unsigned *regno)
+{
+ iter->bits >>= 1;
+ *regno += 1;
+}
+
+#define EXECUTE_IF_SET_IN_HARD_REG_SET(SET, MIN, REGNUM, ITER) \
+ for (hard_reg_set_iter_init (&(ITER), (SET), (MIN), &(REGNUM)); \
+ hard_reg_set_iter_set (&(ITER), &(REGNUM)); \
+ hard_reg_set_iter_next (&(ITER), &(REGNUM)))
+
+
/* Define some standard sets of registers. */
/* Indexed by hard register number, contains 1 for registers
free_INSN_LIST_node (remove_list_elem (elem, listp));
}
+/* Remove and free the first node in the INSN_LIST pointed to by LISTP. */
+rtx
+remove_free_INSN_LIST_node (rtx *listp)
+{
+ rtx node = *listp;
+ rtx elem = XEXP (node, 0);
+
+ remove_list_node (listp);
+ free_INSN_LIST_node (node);
+
+ return elem;
+}
+
+/* Remove and free the first node in the EXPR_LIST pointed to by LISTP. */
+rtx
+remove_free_EXPR_LIST_node (rtx *listp)
+{
+ rtx node = *listp;
+ rtx elem = XEXP (node, 0);
+
+ remove_list_node (listp);
+ free_EXPR_LIST_node (node);
+
+ return elem;
+}
+
#include "gt-lists.h"
/* Create pre-headers. */
if (flags & LOOPS_HAVE_PREHEADERS)
- create_preheaders (CP_SIMPLE_PREHEADERS);
+ {
+ int cp_flags = CP_SIMPLE_PREHEADERS;
+
+ if (flags & LOOPS_HAVE_FALLTHRU_PREHEADERS)
+ cp_flags |= CP_FALLTHRU_PREHEADERS;
+
+ create_preheaders (cp_flags);
+ }
/* Force all latches to have only single successor. */
if (flags & LOOPS_HAVE_SIMPLE_LATCHES)
/* Checking. */
#ifdef ENABLE_CHECKING
- verify_flow_info ();
+ /* FIXME: no point to verify flow info after bundling on ia64. Use this
+ hack for achieving this. */
+ if (!reload_completed)
+ verify_flow_info ();
#endif
}
/* This page defines constants and structures for the modulo scheduling
driver. */
-/* As in haifa-sched.c: */
-/* issue_rate is the number of insns that can be scheduled in the same
- machine cycle. It can be defined in the config/mach/mach.h file,
- otherwise we set it to 1. */
-
-static int issue_rate;
-
static int sms_order_nodes (ddg_ptr, int, int *, int *);
static void set_node_sched_params (ddg_ptr);
static partial_schedule_ptr sms_schedule_by_order (ddg_ptr, int, int, int *);
code in order to use sched_analyze() for computing the dependencies.
They are used when initializing the sched_info structure. */
static const char *
-sms_print_insn (rtx insn, int aligned ATTRIBUTE_UNUSED)
+sms_print_insn (const_rtx insn, int aligned ATTRIBUTE_UNUSED)
{
static char tmp[80];
{
}
-static struct sched_info sms_sched_info =
+static struct common_sched_info_def sms_common_sched_info;
+
+static struct sched_deps_info_def sms_sched_deps_info =
+ {
+ compute_jump_reg_dependencies,
+ NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,
+ NULL,
+ 0, 0, 0
+ };
+
+static struct haifa_sched_info sms_sched_info =
{
NULL,
NULL,
NULL,
sms_print_insn,
NULL,
- compute_jump_reg_dependencies,
NULL, NULL,
NULL, NULL,
- 0, 0, 0,
+ 0, 0,
- NULL, NULL, NULL, NULL, NULL,
+ NULL, NULL, NULL,
0
};
-
/* Given HEAD and TAIL which are the first and last insns in a loop;
return the register which controls the loop. Return zero if it has
more than one occurrence in the loop besides the control part or the
}
}
+/* Setup infos. */
+static void
+setup_sched_infos (void)
+{
+ memcpy (&sms_common_sched_info, &haifa_common_sched_info,
+ sizeof (sms_common_sched_info));
+ sms_common_sched_info.sched_pass_id = SCHED_SMS_PASS;
+ common_sched_info = &sms_common_sched_info;
+
+ sched_deps_info = &sms_sched_deps_info;
+ current_sched_info = &sms_sched_info;
+}
+
/* Probability in % that the sms-ed loop rolls enough so that optimized
version may be entered. Just a guess. */
#define PROB_SMS_ENOUGH_ITERATIONS 80
issue_rate = 1;
/* Initialize the scheduler. */
- current_sched_info = &sms_sched_info;
-
- /* Init Data Flow analysis, to be used in interloop dep calculation. */
- df_set_flags (DF_LR_RUN_DCE);
- df_rd_add_problem ();
- df_note_add_problem ();
- df_chain_add_problem (DF_DU_CHAIN + DF_UD_CHAIN);
- df_analyze ();
- regstat_compute_calls_crossed ();
- sched_init ();
+ setup_sched_infos ();
+ haifa_sched_init ();
/* Allocate memory to hold the DDG array one entry for each loop.
We use loop->num as index into this array. */
free_ddg (g);
}
- regstat_free_calls_crossed ();
free (g_arr);
/* Release scheduler data, needed until now because of DFA. */
- sched_finish ();
+ haifa_sched_finish ();
loop_optimizer_finalize ();
}
unsigned HOST_WIDE_INT g_switch_value;
bool g_switch_set;
+/* Same for selective scheduling. */
+bool sel_sched_switch_set;
+
/* True if we should exit after parsing options. */
bool exit_after_options;
flag_reorder_blocks = 1;
}
+ /* Pipelining of outer loops is only possible when general pipelining
+ capabilities are requested. */
+ if (!flag_sel_sched_pipelining)
+ flag_sel_sched_pipelining_outer_loops = 0;
+
#ifndef IRA_COVER_CLASSES
if (flag_ira)
{
set_random_seed (arg);
break;
+ case OPT_fselective_scheduling:
+ case OPT_fselective_scheduling2:
+ sel_sched_switch_set = true;
+ break;
+
case OPT_fsched_verbose_:
#ifdef INSN_SCHEDULING
fix_sched_param ("verbose", arg);
"The maximum number of insns in a region to be considered for interblock scheduling",
100, 0, 0)
+DEFPARAM(PARAM_MAX_PIPELINE_REGION_BLOCKS,
+ "max-pipeline-region-blocks",
+ "The maximum number of blocks in a region to be considered for interblock scheduling",
+ 15, 0, 0)
+
+DEFPARAM(PARAM_MAX_PIPELINE_REGION_INSNS,
+ "max-pipeline-region-insns",
+ "The maximum number of insns in a region to be considered for interblock scheduling",
+ 200, 0, 0)
+
DEFPARAM(PARAM_MIN_SPEC_PROB,
"min-spec-prob",
"The minimum probability of reaching a source block for interblock speculative scheduling",
"The minimal probability of speculation success (in percents), so that speculative insn will be scheduled.",
40, 0, 100)
+DEFPARAM(PARAM_SELSCHED_MAX_LOOKAHEAD,
+ "selsched-max-lookahead",
+ "The maximum size of the lookahead window of selective scheduling",
+ 50, 0, 0)
+
+DEFPARAM(PARAM_SELSCHED_MAX_SCHED_TIMES,
+ "selsched-max-sched-times",
+ "Maximum number of times that an insn could be scheduled",
+ 2, 0, 0)
+
+DEFPARAM(PARAM_SELSCHED_INSNS_TO_RENAME,
+ "selsched-insns-to-rename",
+ "Maximum number of instructions in the ready list that are considered eligible for renaming",
+ 2, 0, 0)
+
+DEFPARAM (PARAM_SCHED_MEM_TRUE_DEP_COST,
+ "sched-mem-true-dep-cost",
+ "Minimal distance between possibly conflicting store and load",
+ 1, 0, 0)
+
DEFPARAM(PARAM_MAX_LAST_VALUE_RTL,
"max-last-value-rtl",
"The maximum number of RTL nodes that can be recorded as combiner's last value",
}
#endif
-static void validate_replace_rtx_1 (rtx *, rtx, rtx, rtx);
+static void validate_replace_rtx_1 (rtx *, rtx, rtx, rtx, bool);
static void validate_replace_src_1 (rtx *, void *);
static rtx split_insn (rtx);
num_changes = num;
}
-/* Replace every occurrence of FROM in X with TO. Mark each change with
- validate_change passing OBJECT. */
+/* A subroutine of validate_replace_rtx_1 that tries to simplify the resulting
+ rtx. */
static void
-validate_replace_rtx_1 (rtx *loc, rtx from, rtx to, rtx object)
+simplify_while_replacing (rtx *loc, rtx to, rtx object,
+ enum machine_mode op0_mode)
{
- int i, j;
- const char *fmt;
rtx x = *loc;
- enum rtx_code code;
- enum machine_mode op0_mode = VOIDmode;
- int prev_changes = num_changes;
+ enum rtx_code code = GET_CODE (x);
rtx new_rtx;
- if (!x)
- return;
-
- code = GET_CODE (x);
- fmt = GET_RTX_FORMAT (code);
- if (fmt[0] == 'e')
- op0_mode = GET_MODE (XEXP (x, 0));
-
- /* X matches FROM if it is the same rtx or they are both referring to the
- same register in the same mode. Avoid calling rtx_equal_p unless the
- operands look similar. */
-
- if (x == from
- || (REG_P (x) && REG_P (from)
- && GET_MODE (x) == GET_MODE (from)
- && REGNO (x) == REGNO (from))
- || (GET_CODE (x) == GET_CODE (from) && GET_MODE (x) == GET_MODE (from)
- && rtx_equal_p (x, from)))
- {
- validate_unshare_change (object, loc, to, 1);
- return;
- }
-
- /* Call ourself recursively to perform the replacements.
- We must not replace inside already replaced expression, otherwise we
- get infinite recursion for replacements like (reg X)->(subreg (reg X))
- done by regmove, so we must special case shared ASM_OPERANDS. */
-
- if (GET_CODE (x) == PARALLEL)
- {
- for (j = XVECLEN (x, 0) - 1; j >= 0; j--)
- {
- if (j && GET_CODE (XVECEXP (x, 0, j)) == SET
- && GET_CODE (SET_SRC (XVECEXP (x, 0, j))) == ASM_OPERANDS)
- {
- /* Verify that operands are really shared. */
- gcc_assert (ASM_OPERANDS_INPUT_VEC (SET_SRC (XVECEXP (x, 0, 0)))
- == ASM_OPERANDS_INPUT_VEC (SET_SRC (XVECEXP
- (x, 0, j))));
- validate_replace_rtx_1 (&SET_DEST (XVECEXP (x, 0, j)),
- from, to, object);
- }
- else
- validate_replace_rtx_1 (&XVECEXP (x, 0, j), from, to, object);
- }
- }
- else
- for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--)
- {
- if (fmt[i] == 'e')
- validate_replace_rtx_1 (&XEXP (x, i), from, to, object);
- else if (fmt[i] == 'E')
- for (j = XVECLEN (x, i) - 1; j >= 0; j--)
- validate_replace_rtx_1 (&XVECEXP (x, i, j), from, to, object);
- }
-
- /* If we didn't substitute, there is nothing more to do. */
- if (num_changes == prev_changes)
- return;
-
- /* Allow substituted expression to have different mode. This is used by
- regmove to change mode of pseudo register. */
- if (fmt[0] == 'e' && GET_MODE (XEXP (x, 0)) != VOIDmode)
- op0_mode = GET_MODE (XEXP (x, 0));
-
- /* Do changes needed to keep rtx consistent. Don't do any other
- simplifications, as it is not our job. */
-
if (SWAPPABLE_OPERANDS_P (x)
&& swap_commutative_operands_p (XEXP (x, 0), XEXP (x, 1)))
{
}
}
+/* Replace every occurrence of FROM in X with TO. Mark each change with
+ validate_change passing OBJECT. */
+
+static void
+validate_replace_rtx_1 (rtx *loc, rtx from, rtx to, rtx object,
+ bool simplify)
+{
+ int i, j;
+ const char *fmt;
+ rtx x = *loc;
+ enum rtx_code code;
+ enum machine_mode op0_mode = VOIDmode;
+ int prev_changes = num_changes;
+
+ if (!x)
+ return;
+
+ code = GET_CODE (x);
+ fmt = GET_RTX_FORMAT (code);
+ if (fmt[0] == 'e')
+ op0_mode = GET_MODE (XEXP (x, 0));
+
+ /* X matches FROM if it is the same rtx or they are both referring to the
+ same register in the same mode. Avoid calling rtx_equal_p unless the
+ operands look similar. */
+
+ if (x == from
+ || (REG_P (x) && REG_P (from)
+ && GET_MODE (x) == GET_MODE (from)
+ && REGNO (x) == REGNO (from))
+ || (GET_CODE (x) == GET_CODE (from) && GET_MODE (x) == GET_MODE (from)
+ && rtx_equal_p (x, from)))
+ {
+ validate_unshare_change (object, loc, to, 1);
+ return;
+ }
+
+ /* Call ourself recursively to perform the replacements.
+ We must not replace inside already replaced expression, otherwise we
+ get infinite recursion for replacements like (reg X)->(subreg (reg X))
+ done by regmove, so we must special case shared ASM_OPERANDS. */
+
+ if (GET_CODE (x) == PARALLEL)
+ {
+ for (j = XVECLEN (x, 0) - 1; j >= 0; j--)
+ {
+ if (j && GET_CODE (XVECEXP (x, 0, j)) == SET
+ && GET_CODE (SET_SRC (XVECEXP (x, 0, j))) == ASM_OPERANDS)
+ {
+ /* Verify that operands are really shared. */
+ gcc_assert (ASM_OPERANDS_INPUT_VEC (SET_SRC (XVECEXP (x, 0, 0)))
+ == ASM_OPERANDS_INPUT_VEC (SET_SRC (XVECEXP
+ (x, 0, j))));
+ validate_replace_rtx_1 (&SET_DEST (XVECEXP (x, 0, j)),
+ from, to, object, simplify);
+ }
+ else
+ validate_replace_rtx_1 (&XVECEXP (x, 0, j), from, to, object,
+ simplify);
+ }
+ }
+ else
+ for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--)
+ {
+ if (fmt[i] == 'e')
+ validate_replace_rtx_1 (&XEXP (x, i), from, to, object, simplify);
+ else if (fmt[i] == 'E')
+ for (j = XVECLEN (x, i) - 1; j >= 0; j--)
+ validate_replace_rtx_1 (&XVECEXP (x, i, j), from, to, object,
+ simplify);
+ }
+
+ /* If we didn't substitute, there is nothing more to do. */
+ if (num_changes == prev_changes)
+ return;
+
+ /* Allow substituted expression to have different mode. This is used by
+ regmove to change mode of pseudo register. */
+ if (fmt[0] == 'e' && GET_MODE (XEXP (x, 0)) != VOIDmode)
+ op0_mode = GET_MODE (XEXP (x, 0));
+
+ /* Do changes needed to keep rtx consistent. Don't do any other
+ simplifications, as it is not our job. */
+ if (simplify)
+ simplify_while_replacing (loc, to, object, op0_mode);
+}
+
/* Try replacing every occurrence of FROM in INSN with TO. After all
changes have been made, validate by seeing if INSN is still valid. */
int
validate_replace_rtx (rtx from, rtx to, rtx insn)
{
- validate_replace_rtx_1 (&PATTERN (insn), from, to, insn);
+ validate_replace_rtx_1 (&PATTERN (insn), from, to, insn, true);
return apply_change_group ();
}
+/* Try replacing every occurrence of FROM in WHERE with TO. Assume that WHERE
+ is a part of INSN. After all changes have been made, validate by seeing if
+ INSN is still valid.
+ validate_replace_rtx (from, to, insn) is equivalent to
+ validate_replace_rtx_part (from, to, &PATTERN (insn), insn). */
+
+int
+validate_replace_rtx_part (rtx from, rtx to, rtx *where, rtx insn)
+{
+ validate_replace_rtx_1 (where, from, to, insn, true);
+ return apply_change_group ();
+}
+
+/* Same as above, but do not simplify rtx afterwards. */
+int
+validate_replace_rtx_part_nosimplify (rtx from, rtx to, rtx *where,
+ rtx insn)
+{
+ validate_replace_rtx_1 (where, from, to, insn, false);
+ return apply_change_group ();
+
+}
+
/* Try replacing every occurrence of FROM in INSN with TO. */
void
validate_replace_rtx_group (rtx from, rtx to, rtx insn)
{
- validate_replace_rtx_1 (&PATTERN (insn), from, to, insn);
+ validate_replace_rtx_1 (&PATTERN (insn), from, to, insn, true);
}
/* Function called by note_uses to replace used subexpressions. */
struct validate_replace_src_data *d
= (struct validate_replace_src_data *) data;
- validate_replace_rtx_1 (x, d->from, d->to, d->insn);
+ validate_replace_rtx_1 (x, d->from, d->to, d->insn, true);
}
/* Try replacing every occurrence of FROM in INSN with TO, avoiding
extern int memory_address_p (enum machine_mode, rtx);
extern int strict_memory_address_p (enum machine_mode, rtx);
extern int validate_replace_rtx (rtx, rtx, rtx);
+extern int validate_replace_rtx_part (rtx, rtx, rtx *, rtx);
+extern int validate_replace_rtx_part_nosimplify (rtx, rtx, rtx *, rtx);
extern void validate_replace_rtx_group (rtx, rtx, rtx);
extern void validate_replace_src_group (rtx, rtx, rtx);
extern bool validate_simplify_insn (rtx insn);
int currently_expanding_to_rtl;
\f
-/* Return 1 if X and Y are identical-looking rtx's.
- This is the Lisp function EQUAL for rtx arguments. */
+
+/* Same as rtx_equal_p, but call CB on each pair of rtx if CB is not NULL.
+ When the callback returns true, we continue with the new pair. */
int
-rtx_equal_p (const_rtx x, const_rtx y)
+rtx_equal_p_cb (const_rtx x, const_rtx y, rtx_equal_p_callback_function cb)
{
int i;
int j;
enum rtx_code code;
const char *fmt;
+ rtx nx, ny;
if (x == y)
return 1;
if (x == 0 || y == 0)
return 0;
+ /* Invoke the callback first. */
+ if (cb != NULL
+ && ((*cb) (&x, &y, &nx, &ny)))
+ return rtx_equal_p_cb (nx, ny, cb);
+
code = GET_CODE (x);
/* Rtx's of different codes cannot be equal. */
if (code != GET_CODE (y))
/* And the corresponding elements must match. */
for (j = 0; j < XVECLEN (x, i); j++)
- if (rtx_equal_p (XVECEXP (x, i, j), XVECEXP (y, i, j)) == 0)
+ if (rtx_equal_p_cb (XVECEXP (x, i, j),
+ XVECEXP (y, i, j), cb) == 0)
return 0;
break;
case 'e':
- if (rtx_equal_p (XEXP (x, i), XEXP (y, i)) == 0)
+ if (rtx_equal_p_cb (XEXP (x, i), XEXP (y, i), cb) == 0)
return 0;
break;
return 1;
}
+/* Return 1 if X and Y are identical-looking rtx's.
+ This is the Lisp function EQUAL for rtx arguments. */
+
+int
+rtx_equal_p (const_rtx x, const_rtx y)
+{
+ return rtx_equal_p_cb (x, y, NULL);
+}
+
void
dump_rtx_statistics (void)
{
extern int rtx_referenced_p (rtx, rtx);
extern bool tablejump_p (const_rtx, rtx *, rtx *);
extern int computed_jump_p (const_rtx);
+
typedef int (*rtx_function) (rtx *, void *);
extern int for_each_rtx (rtx *, rtx_function, void *);
+
+typedef int (*rtx_equal_p_callback_function) (const_rtx *, const_rtx *,
+ rtx *, rtx *);
+extern int rtx_equal_p_cb (const_rtx, const_rtx,
+ rtx_equal_p_callback_function);
+
+typedef int (*hash_rtx_callback_function) (const_rtx, enum machine_mode, rtx *,
+ enum machine_mode *);
+extern unsigned hash_rtx_cb (const_rtx, enum machine_mode, int *, int *,
+ bool, hash_rtx_callback_function);
+
extern rtx regno_use_in (unsigned int, rtx);
extern int auto_inc_p (const_rtx);
extern int in_expr_list_p (const_rtx, const_rtx);
/* lists.c */
-void free_EXPR_LIST_list (rtx *);
-void free_INSN_LIST_list (rtx *);
-void free_EXPR_LIST_node (rtx);
-void free_INSN_LIST_node (rtx);
-rtx alloc_INSN_LIST (rtx, rtx);
-rtx alloc_EXPR_LIST (int, rtx, rtx);
-void remove_free_INSN_LIST_elem (rtx, rtx *);
-rtx remove_list_elem (rtx, rtx *);
+extern void free_EXPR_LIST_list (rtx *);
+extern void free_INSN_LIST_list (rtx *);
+extern void free_EXPR_LIST_node (rtx);
+extern void free_INSN_LIST_node (rtx);
+extern rtx alloc_INSN_LIST (rtx, rtx);
+extern rtx alloc_EXPR_LIST (int, rtx, rtx);
+extern void remove_free_INSN_LIST_elem (rtx, rtx *);
+extern rtx remove_list_elem (rtx, rtx *);
+extern rtx remove_free_INSN_LIST_node (rtx *);
+extern rtx remove_free_EXPR_LIST_node (rtx *);
+
/* regclass.c */
extern void delete_dead_jumptables (void);
/* In sched-vis.c. */
-extern void print_insn (char *, rtx, int);
+extern void debug_bb_n_slim (int);
+extern void debug_bb_slim (struct basic_block_def *);
+extern void print_rtl_slim (FILE *, rtx, rtx, int, int);
extern void print_rtl_slim_with_bb (FILE *, rtx, int);
extern void dump_insn_slim (FILE *f, rtx x);
extern void debug_insn_slim (rtx x);
/* In haifa-sched.c. */
extern void fix_sched_param (const char *, const char *);
+/* In sel-sched-dump.c. */
+extern void sel_sched_fix_param (const char *param, const char *val);
+
/* In print-rtl.c */
extern const char *print_rtx_head;
extern void debug_rtx (const_rtx);
/* In loop-iv.c */
extern rtx canon_condition (rtx);
extern void simplify_using_condition (rtx, rtx *, struct bitmap_head_def *);
+
+/* In final.c */
+extern unsigned int compute_alignments (void);
\f
struct rtl_hooks
{
RTL_HOOKS_GEN_LOWPART_NO_EMIT, \
RTL_HOOKS_REG_NONZERO_REG_BITS, \
RTL_HOOKS_REG_NUM_SIGN_BIT_COPIES, \
- RTL_HOOKS_REG_TRUNCATED_TO_MODE, \
+ RTL_HOOKS_REG_TRUNCATED_TO_MODE \
}
extern rtx gen_lowpart_general (enum machine_mode, rtx);
#define CHECK (false)
#endif
+/* Holds current parameters for the dependency analyzer. */
+struct sched_deps_info_def *sched_deps_info;
+
+/* The data is specific to the Haifa scheduler. */
+VEC(haifa_deps_insn_data_def, heap) *h_d_i_d = NULL;
+
/* Return the major type present in the DS. */
enum reg_note
ds_to_dk (ds_t ds)
static regset reg_pending_sets;
static regset reg_pending_clobbers;
static regset reg_pending_uses;
-
-/* The following enumeration values tell us what dependencies we
- should use to implement the barrier. We use true-dependencies for
- TRUE_BARRIER and anti-dependencies for MOVE_BARRIER. */
-enum reg_pending_barrier_mode
-{
- NOT_A_BARRIER = 0,
- MOVE_BARRIER,
- TRUE_BARRIER
-};
-
static enum reg_pending_barrier_mode reg_pending_barrier;
/* To speed up the test for duplicate dependency links we keep a
has enough entries to represent a dependency on any other insn in
the insn chain. All bitmap for true dependencies cache is
allocated then the rest two ones are also allocated. */
-static bitmap_head *true_dependency_cache;
-static bitmap_head *output_dependency_cache;
-static bitmap_head *anti_dependency_cache;
-static bitmap_head *spec_dependency_cache;
+static bitmap_head *true_dependency_cache = NULL;
+static bitmap_head *output_dependency_cache = NULL;
+static bitmap_head *anti_dependency_cache = NULL;
+static bitmap_head *spec_dependency_cache = NULL;
static int cache_size;
static int deps_may_trap_p (const_rtx);
static void add_dependence_list (rtx, rtx, int, enum reg_note);
-static void add_dependence_list_and_free (rtx, rtx *, int, enum reg_note);
+static void add_dependence_list_and_free (struct deps *, rtx,
+ rtx *, int, enum reg_note);
static void delete_all_dependences (rtx);
static void fixup_sched_groups (rtx);
static void sched_analyze_2 (struct deps *, rtx, rtx);
static void sched_analyze_insn (struct deps *, rtx, rtx);
-static rtx sched_get_condition (const_rtx);
-static int conditions_mutex_p (const_rtx, const_rtx);
+static bool sched_has_condition_p (const_rtx);
+static int conditions_mutex_p (const_rtx, const_rtx, bool, bool);
static enum DEPS_ADJUST_RESULT maybe_add_or_update_dep_1 (dep_t, bool,
rtx, rtx);
static enum DEPS_ADJUST_RESULT add_or_update_dep_1 (dep_t, bool, rtx, rtx);
-static dw_t estimate_dep_weak (rtx, rtx);
#ifdef ENABLE_CHECKING
static void check_dep (dep_t, bool);
#endif
return rtx_addr_can_trap_p (addr);
}
\f
-/* Find the condition under which INSN is executed. */
+/* Find the condition under which INSN is executed. If REV is not NULL,
+ it is set to TRUE when the returned comparison should be reversed
+ to get the actual condition. */
static rtx
-sched_get_condition (const_rtx insn)
+sched_get_condition_with_rev (const_rtx insn, bool *rev)
{
rtx pat = PATTERN (insn);
rtx src;
if (pat == 0)
return 0;
+ if (rev)
+ *rev = false;
+
if (GET_CODE (pat) == COND_EXEC)
return COND_EXEC_TEST (pat);
if (revcode == UNKNOWN)
return 0;
- return gen_rtx_fmt_ee (revcode, GET_MODE (cond), XEXP (cond, 0),
- XEXP (cond, 1));
+
+ if (rev)
+ *rev = true;
+ return cond;
}
return 0;
}
+/* True when we can find a condition under which INSN is executed. */
+static bool
+sched_has_condition_p (const_rtx insn)
+{
+ return !! sched_get_condition_with_rev (insn, NULL);
+}
+
\f
-/* Return nonzero if conditions COND1 and COND2 can never be both true. */
+/* Return nonzero if conditions COND1 and COND2 can never be both true. */
static int
-conditions_mutex_p (const_rtx cond1, const_rtx cond2)
+conditions_mutex_p (const_rtx cond1, const_rtx cond2, bool rev1, bool rev2)
{
if (COMPARISON_P (cond1)
&& COMPARISON_P (cond2)
- && GET_CODE (cond1) == reversed_comparison_code (cond2, NULL)
+ && GET_CODE (cond1) ==
+ (rev1==rev2
+ ? reversed_comparison_code (cond2, NULL)
+ : GET_CODE (cond2))
&& XEXP (cond1, 0) == XEXP (cond2, 0)
&& XEXP (cond1, 1) == XEXP (cond2, 1))
return 1;
sched_insns_conditions_mutex_p (const_rtx insn1, const_rtx insn2)
{
rtx cond1, cond2;
+ bool rev1, rev2;
/* df doesn't handle conditional lifetimes entirely correctly;
calls mess up the conditional lifetimes. */
if (!CALL_P (insn1) && !CALL_P (insn2))
{
- cond1 = sched_get_condition (insn1);
- cond2 = sched_get_condition (insn2);
+ cond1 = sched_get_condition_with_rev (insn1, &rev1);
+ cond2 = sched_get_condition_with_rev (insn2, &rev2);
if (cond1 && cond2
- && conditions_mutex_p (cond1, cond2)
+ && conditions_mutex_p (cond1, cond2, rev1, rev2)
/* Make sure first instruction doesn't affect condition of second
instruction if switched. */
&& !modified_in_p (cond1, insn2)
if (SCHED_GROUP_P (insn))
return false;
- if (IS_SPECULATION_CHECK_P (insn))
+ if (IS_SPECULATION_CHECK_P (CONST_CAST_RTX (insn)))
return false;
if (side_effects_p (PATTERN (insn)))
return false;
if ((ds & BE_IN_DATA)
- && sched_get_condition (insn) != NULL_RTX)
+ && sched_has_condition_p (insn))
/* If this is a predicated instruction, then it cannot be
speculatively scheduled. See PR35659. */
return false;
/* Don't depend an insn on itself. */
if (insn == elem)
{
- if (current_sched_info->flags & DO_SPECULATION)
+ if (sched_deps_info->generate_spec_deps)
/* INSN has an internal dependence, which we can't overcome. */
HAS_INTERNAL_DEP (insn) = 1;
if (mem1 != NULL_RTX)
{
- gcc_assert (current_sched_info->flags & DO_SPECULATION);
+ gcc_assert (sched_deps_info->generate_spec_deps);
DEP_STATUS (new_dep) = set_dep_weak (DEP_STATUS (new_dep), BEGIN_DATA,
estimate_dep_weak (mem1, mem2));
}
}
}
-/* Similar, but free *LISTP at the same time. */
+/* Similar, but free *LISTP at the same time, when the context
+ is not readonly. */
static void
-add_dependence_list_and_free (rtx insn, rtx *listp, int uncond,
- enum reg_note dep_type)
+add_dependence_list_and_free (struct deps *deps, rtx insn, rtx *listp,
+ int uncond, enum reg_note dep_type)
{
rtx list, next;
+
+ if (deps->readonly)
+ {
+ add_dependence_list (insn, *listp, uncond, dep_type);
+ return;
+ }
+
for (list = *listp, *listp = NULL; list ; list = next)
{
next = XEXP (list, 1);
}
}
+/* Remove all occurences of INSN from LIST. Return the number of
+ occurences removed. */
+
+static int
+remove_from_dependence_list (rtx insn, rtx* listp)
+{
+ int removed = 0;
+
+ while (*listp)
+ {
+ if (XEXP (*listp, 0) == insn)
+ {
+ remove_free_INSN_LIST_node (listp);
+ removed++;
+ continue;
+ }
+
+ listp = &XEXP (*listp, 1);
+ }
+
+ return removed;
+}
+
+/* Same as above, but process two lists at once. */
+static int
+remove_from_both_dependence_lists (rtx insn, rtx *listp, rtx *exprp)
+{
+ int removed = 0;
+
+ while (*listp)
+ {
+ if (XEXP (*listp, 0) == insn)
+ {
+ remove_free_INSN_LIST_node (listp);
+ remove_free_EXPR_LIST_node (exprp);
+ removed++;
+ continue;
+ }
+
+ listp = &XEXP (*listp, 1);
+ exprp = &XEXP (*exprp, 1);
+ }
+
+ return removed;
+}
+
/* Clear all dependencies for an insn. */
static void
delete_all_dependences (rtx insn)
rtx *mem_list;
rtx link;
+ gcc_assert (!deps->readonly);
if (read_p)
{
insn_list = &deps->pending_read_insns;
link = alloc_INSN_LIST (insn, *insn_list);
*insn_list = link;
- if (current_sched_info->use_cselib)
+ if (sched_deps_info->use_cselib)
{
mem = shallow_copy_rtx (mem);
XEXP (mem, 0) = cselib_subst_to_values (XEXP (mem, 0));
{
if (for_write)
{
- add_dependence_list_and_free (insn, &deps->pending_read_insns, 1,
- REG_DEP_ANTI);
- free_EXPR_LIST_list (&deps->pending_read_mems);
- deps->pending_read_list_length = 0;
+ add_dependence_list_and_free (deps, insn, &deps->pending_read_insns,
+ 1, REG_DEP_ANTI);
+ if (!deps->readonly)
+ {
+ free_EXPR_LIST_list (&deps->pending_read_mems);
+ deps->pending_read_list_length = 0;
+ }
}
- add_dependence_list_and_free (insn, &deps->pending_write_insns, 1,
+ add_dependence_list_and_free (deps, insn, &deps->pending_write_insns, 1,
for_read ? REG_DEP_ANTI : REG_DEP_OUTPUT);
- free_EXPR_LIST_list (&deps->pending_write_mems);
- deps->pending_write_list_length = 0;
- add_dependence_list_and_free (insn, &deps->last_pending_memory_flush, 1,
- for_read ? REG_DEP_ANTI : REG_DEP_OUTPUT);
- deps->last_pending_memory_flush = alloc_INSN_LIST (insn, NULL_RTX);
- deps->pending_flush_length = 1;
+ add_dependence_list_and_free (deps, insn,
+ &deps->last_pending_memory_flush, 1,
+ for_read ? REG_DEP_ANTI : REG_DEP_OUTPUT);
+ if (!deps->readonly)
+ {
+ free_EXPR_LIST_list (&deps->pending_write_mems);
+ deps->pending_write_list_length = 0;
+
+ deps->last_pending_memory_flush = alloc_INSN_LIST (insn, NULL_RTX);
+ deps->pending_flush_length = 1;
+ }
+}
+\f
+/* Instruction which dependencies we are analyzing. */
+static rtx cur_insn = NULL_RTX;
+
+/* Implement hooks for haifa scheduler. */
+
+static void
+haifa_start_insn (rtx insn)
+{
+ gcc_assert (insn && !cur_insn);
+
+ cur_insn = insn;
+}
+
+static void
+haifa_finish_insn (void)
+{
+ cur_insn = NULL;
+}
+
+void
+haifa_note_reg_set (int regno)
+{
+ SET_REGNO_REG_SET (reg_pending_sets, regno);
+}
+
+void
+haifa_note_reg_clobber (int regno)
+{
+ SET_REGNO_REG_SET (reg_pending_clobbers, regno);
+}
+
+void
+haifa_note_reg_use (int regno)
+{
+ SET_REGNO_REG_SET (reg_pending_uses, regno);
+}
+
+static void
+haifa_note_mem_dep (rtx mem, rtx pending_mem, rtx pending_insn, ds_t ds)
+{
+ if (!(ds & SPECULATIVE))
+ {
+ mem = NULL_RTX;
+ pending_mem = NULL_RTX;
+ }
+ else
+ gcc_assert (ds & BEGIN_DATA);
+
+ {
+ dep_def _dep, *dep = &_dep;
+
+ init_dep_1 (dep, pending_insn, cur_insn, ds_to_dt (ds),
+ current_sched_info->flags & USE_DEPS_LIST ? ds : -1);
+ maybe_add_or_update_dep_1 (dep, false, pending_mem, mem);
+ }
+
+}
+
+static void
+haifa_note_dep (rtx elem, ds_t ds)
+{
+ dep_def _dep;
+ dep_t dep = &_dep;
+
+ init_dep (dep, elem, cur_insn, ds_to_dt (ds));
+ maybe_add_or_update_dep_1 (dep, false, NULL_RTX, NULL_RTX);
+}
+
+static void
+note_reg_use (int r)
+{
+ if (sched_deps_info->note_reg_use)
+ sched_deps_info->note_reg_use (r);
+}
+
+static void
+note_reg_set (int r)
+{
+ if (sched_deps_info->note_reg_set)
+ sched_deps_info->note_reg_set (r);
+}
+
+static void
+note_reg_clobber (int r)
+{
+ if (sched_deps_info->note_reg_clobber)
+ sched_deps_info->note_reg_clobber (r);
+}
+
+static void
+note_mem_dep (rtx m1, rtx m2, rtx e, ds_t ds)
+{
+ if (sched_deps_info->note_mem_dep)
+ sched_deps_info->note_mem_dep (m1, m2, e, ds);
+}
+
+static void
+note_dep (rtx e, ds_t ds)
+{
+ if (sched_deps_info->note_dep)
+ sched_deps_info->note_dep (e, ds);
+}
+
+/* Return corresponding to DS reg_note. */
+enum reg_note
+ds_to_dt (ds_t ds)
+{
+ if (ds & DEP_TRUE)
+ return REG_DEP_TRUE;
+ else if (ds & DEP_OUTPUT)
+ return REG_DEP_OUTPUT;
+ else
+ {
+ gcc_assert (ds & DEP_ANTI);
+ return REG_DEP_ANTI;
+ }
}
\f
+
+/* Internal variable for sched_analyze_[12] () functions.
+ If it is nonzero, this means that sched_analyze_[12] looks
+ at the most toplevel SET. */
+static bool can_start_lhs_rhs_p;
+
+/* Extend reg info for the deps context DEPS given that
+ we have just generated a register numbered REGNO. */
+static void
+extend_deps_reg_info (struct deps *deps, int regno)
+{
+ int max_regno = regno + 1;
+
+ gcc_assert (!reload_completed);
+
+ /* In a readonly context, it would not hurt to extend info,
+ but it should not be needed. */
+ if (reload_completed && deps->readonly)
+ {
+ deps->max_reg = max_regno;
+ return;
+ }
+
+ if (max_regno > deps->max_reg)
+ {
+ deps->reg_last = XRESIZEVEC (struct deps_reg, deps->reg_last,
+ max_regno);
+ memset (&deps->reg_last[deps->max_reg],
+ 0, (max_regno - deps->max_reg)
+ * sizeof (struct deps_reg));
+ deps->max_reg = max_regno;
+ }
+}
+
+/* Extends REG_INFO_P if needed. */
+void
+maybe_extend_reg_info_p (void)
+{
+ /* Extend REG_INFO_P, if needed. */
+ if ((unsigned int)max_regno - 1 >= reg_info_p_size)
+ {
+ size_t new_reg_info_p_size = max_regno + 128;
+
+ gcc_assert (!reload_completed && sel_sched_p ());
+
+ reg_info_p = (struct reg_info_t *) xrecalloc (reg_info_p,
+ new_reg_info_p_size,
+ reg_info_p_size,
+ sizeof (*reg_info_p));
+ reg_info_p_size = new_reg_info_p_size;
+ }
+}
+
/* Analyze a single reference to register (reg:MODE REGNO) in INSN.
The type of the reference is specified by REF and can be SET,
CLOBBER, PRE_DEC, POST_DEC, PRE_INC, POST_INC or USE. */
sched_analyze_reg (struct deps *deps, int regno, enum machine_mode mode,
enum rtx_code ref, rtx insn)
{
+ /* We could emit new pseudos in renaming. Extend the reg structures. */
+ if (!reload_completed && sel_sched_p ()
+ && (regno >= max_reg_num () - 1 || regno >= deps->max_reg))
+ extend_deps_reg_info (deps, regno);
+
+ maybe_extend_reg_info_p ();
+
/* A hard reg in a wide mode may really be multiple registers.
If so, mark all of them just like the first. */
if (regno < FIRST_PSEUDO_REGISTER)
if (ref == SET)
{
while (--i >= 0)
- SET_REGNO_REG_SET (reg_pending_sets, regno + i);
+ note_reg_set (regno + i);
}
else if (ref == USE)
{
while (--i >= 0)
- SET_REGNO_REG_SET (reg_pending_uses, regno + i);
+ note_reg_use (regno + i);
}
else
{
while (--i >= 0)
- SET_REGNO_REG_SET (reg_pending_clobbers, regno + i);
+ note_reg_clobber (regno + i);
}
}
else
{
if (ref == SET)
- SET_REGNO_REG_SET (reg_pending_sets, regno);
+ note_reg_set (regno);
else if (ref == USE)
- SET_REGNO_REG_SET (reg_pending_uses, regno);
+ note_reg_use (regno);
else
- SET_REGNO_REG_SET (reg_pending_clobbers, regno);
+ note_reg_clobber (regno);
/* Pseudos that are REG_EQUIV to something may be replaced
by that during reloading. We need only add dependencies for
already cross one. */
if (REG_N_CALLS_CROSSED (regno) == 0)
{
- if (ref == USE)
+ if (!deps->readonly
+ && ref == USE)
deps->sched_before_next_call
= alloc_INSN_LIST (insn, deps->sched_before_next_call);
else
{
rtx dest = XEXP (x, 0);
enum rtx_code code = GET_CODE (x);
+ bool cslr_p = can_start_lhs_rhs_p;
+ can_start_lhs_rhs_p = false;
+
+ gcc_assert (dest);
if (dest == 0)
return;
+ if (cslr_p && sched_deps_info->start_lhs)
+ sched_deps_info->start_lhs (dest);
+
if (GET_CODE (dest) == PARALLEL)
{
int i;
XEXP (XVECEXP (dest, 0, i), 0)),
insn);
- if (GET_CODE (x) == SET)
- sched_analyze_2 (deps, SET_SRC (x), insn);
+ if (cslr_p && sched_deps_info->finish_lhs)
+ sched_deps_info->finish_lhs ();
+
+ if (code == SET)
+ {
+ can_start_lhs_rhs_p = cslr_p;
+
+ sched_analyze_2 (deps, SET_SRC (x), insn);
+
+ can_start_lhs_rhs_p = false;
+ }
+
return;
}
/* Writing memory. */
rtx t = dest;
- if (current_sched_info->use_cselib)
+ if (sched_deps_info->use_cselib)
{
t = shallow_copy_rtx (dest);
cselib_lookup (XEXP (t, 0), Pmode, 1);
}
t = canon_rtx (t);
- if ((deps->pending_read_list_length + deps->pending_write_list_length)
- > MAX_PENDING_LIST_LENGTH)
+ /* Pending lists can't get larger with a readonly context. */
+ if (!deps->readonly
+ && ((deps->pending_read_list_length + deps->pending_write_list_length)
+ > MAX_PENDING_LIST_LENGTH))
{
/* Flush all pending reads and writes to prevent the pending lists
from getting any larger. Insn scheduling runs too slowly when
{
if (anti_dependence (XEXP (pending_mem, 0), t)
&& ! sched_insns_conditions_mutex_p (insn, XEXP (pending, 0)))
- add_dependence (insn, XEXP (pending, 0), REG_DEP_ANTI);
+ note_mem_dep (t, XEXP (pending_mem, 0), XEXP (pending, 0),
+ DEP_ANTI);
pending = XEXP (pending, 1);
pending_mem = XEXP (pending_mem, 1);
{
if (output_dependence (XEXP (pending_mem, 0), t)
&& ! sched_insns_conditions_mutex_p (insn, XEXP (pending, 0)))
- add_dependence (insn, XEXP (pending, 0), REG_DEP_OUTPUT);
+ note_mem_dep (t, XEXP (pending_mem, 0), XEXP (pending, 0),
+ DEP_OUTPUT);
pending = XEXP (pending, 1);
pending_mem = XEXP (pending_mem, 1);
add_dependence_list (insn, deps->last_pending_memory_flush, 1,
REG_DEP_ANTI);
- add_insn_mem_dependence (deps, false, insn, dest);
+ if (!deps->readonly)
+ add_insn_mem_dependence (deps, false, insn, dest);
}
sched_analyze_2 (deps, XEXP (dest, 0), insn);
}
+ if (cslr_p && sched_deps_info->finish_lhs)
+ sched_deps_info->finish_lhs ();
+
/* Analyze reads. */
if (GET_CODE (x) == SET)
- sched_analyze_2 (deps, SET_SRC (x), insn);
+ {
+ can_start_lhs_rhs_p = cslr_p;
+
+ sched_analyze_2 (deps, SET_SRC (x), insn);
+
+ can_start_lhs_rhs_p = false;
+ }
}
/* Analyze the uses of memory and registers in rtx X in INSN. */
-
static void
sched_analyze_2 (struct deps *deps, rtx x, rtx insn)
{
int j;
enum rtx_code code;
const char *fmt;
+ bool cslr_p = can_start_lhs_rhs_p;
+
+ can_start_lhs_rhs_p = false;
+ gcc_assert (x);
if (x == 0)
return;
+ if (cslr_p && sched_deps_info->start_rhs)
+ sched_deps_info->start_rhs (x);
+
code = GET_CODE (x);
switch (code)
/* Ignore constants. Note that we must handle CONST_DOUBLE here
because it may have a cc0_rtx in its CONST_DOUBLE_CHAIN field, but
this does not mean that this insn is using cc0. */
+
+ if (cslr_p && sched_deps_info->finish_rhs)
+ sched_deps_info->finish_rhs ();
+
return;
#ifdef HAVE_cc0
/* Don't move CC0 setter to another block (it can set up the
same flag for previous CC0 users which is safe). */
CANT_MOVE (prev_nonnote_insn (insn)) = 1;
+
+ if (cslr_p && sched_deps_info->finish_rhs)
+ sched_deps_info->finish_rhs ();
+
return;
#endif
sched_analyze_reg (deps, FIRST_STACK_REG, mode, SET, insn);
}
#endif
+
+ if (cslr_p && sched_deps_info->finish_rhs)
+ sched_deps_info->finish_rhs ();
+
return;
}
rtx pending, pending_mem;
rtx t = x;
- if (current_sched_info->use_cselib)
+ if (sched_deps_info->use_cselib)
{
t = shallow_copy_rtx (t);
cselib_lookup (XEXP (t, 0), Pmode, 1);
{
if (read_dependence (XEXP (pending_mem, 0), t)
&& ! sched_insns_conditions_mutex_p (insn, XEXP (pending, 0)))
- add_dependence (insn, XEXP (pending, 0), REG_DEP_ANTI);
+ note_mem_dep (t, XEXP (pending_mem, 0), XEXP (pending, 0),
+ DEP_ANTI);
pending = XEXP (pending, 1);
pending_mem = XEXP (pending_mem, 1);
if (true_dependence (XEXP (pending_mem, 0), VOIDmode,
t, rtx_varies_p)
&& ! sched_insns_conditions_mutex_p (insn, XEXP (pending, 0)))
- {
- if ((current_sched_info->flags & DO_SPECULATION)
- && (spec_info->mask & BEGIN_DATA))
- /* Create a data-speculative dependence between producer
- and consumer. */
- {
- dep_def _dep, *dep = &_dep;
-
- init_dep_1 (dep, XEXP (pending, 0), insn, REG_DEP_TRUE,
- BEGIN_DATA | DEP_TRUE);
-
- maybe_add_or_update_dep_1 (dep, false,
- XEXP (pending_mem, 0), t);
- }
- else
- add_dependence (insn, XEXP (pending, 0), REG_DEP_TRUE);
- }
+ note_mem_dep (t, XEXP (pending_mem, 0), XEXP (pending, 0),
+ sched_deps_info->generate_spec_deps
+ ? BEGIN_DATA | DEP_TRUE : DEP_TRUE);
pending = XEXP (pending, 1);
pending_mem = XEXP (pending_mem, 1);
}
for (u = deps->last_pending_memory_flush; u; u = XEXP (u, 1))
- if (! JUMP_P (XEXP (u, 0)) || deps_may_trap_p (x))
- add_dependence (insn, XEXP (u, 0), REG_DEP_ANTI);
+ {
+ if (! JUMP_P (XEXP (u, 0)))
+ add_dependence (insn, XEXP (u, 0), REG_DEP_ANTI);
+ else if (deps_may_trap_p (x))
+ {
+ if ((sched_deps_info->generate_spec_deps)
+ && sel_sched_p () && (spec_info->mask & BEGIN_CONTROL))
+ {
+ ds_t ds = set_dep_weak (DEP_ANTI, BEGIN_CONTROL,
+ MAX_DEP_WEAK);
+
+ note_dep (XEXP (u, 0), ds);
+ }
+ else
+ add_dependence (insn, XEXP (u, 0), REG_DEP_ANTI);
+ }
+ }
/* Always add these dependencies to pending_reads, since
this insn may be followed by a write. */
- add_insn_mem_dependence (deps, true, insn, x);
+ if (!deps->readonly)
+ add_insn_mem_dependence (deps, true, insn, x);
/* Take advantage of tail recursion here. */
sched_analyze_2 (deps, XEXP (x, 0), insn);
+
+ if (cslr_p && sched_deps_info->finish_rhs)
+ sched_deps_info->finish_rhs ();
+
return;
}
{
for (j = 0; j < ASM_OPERANDS_INPUT_LENGTH (x); j++)
sched_analyze_2 (deps, ASM_OPERANDS_INPUT (x, j), insn);
+
+ if (cslr_p && sched_deps_info->finish_rhs)
+ sched_deps_info->finish_rhs ();
+
return;
}
break;
to get the proper antecedent for the read. */
sched_analyze_2 (deps, XEXP (x, 0), insn);
sched_analyze_1 (deps, x, insn);
+
+ if (cslr_p && sched_deps_info->finish_rhs)
+ sched_deps_info->finish_rhs ();
+
return;
case POST_MODIFY:
sched_analyze_2 (deps, XEXP (x, 0), insn);
sched_analyze_2 (deps, XEXP (x, 1), insn);
sched_analyze_1 (deps, x, insn);
+
+ if (cslr_p && sched_deps_info->finish_rhs)
+ sched_deps_info->finish_rhs ();
+
return;
default:
for (j = 0; j < XVECLEN (x, i); j++)
sched_analyze_2 (deps, XVECEXP (x, i, j), insn);
}
+
+ if (cslr_p && sched_deps_info->finish_rhs)
+ sched_deps_info->finish_rhs ();
}
/* Analyze an INSN with pattern X to find all dependencies. */
-
static void
sched_analyze_insn (struct deps *deps, rtx x, rtx insn)
{
unsigned i;
reg_set_iterator rsi;
+ can_start_lhs_rhs_p = (NONJUMP_INSN_P (insn)
+ && code == SET);
+
if (code == COND_EXEC)
{
sched_analyze_2 (deps, COND_EXEC_TEST (x), insn);
else
{
rtx pending, pending_mem;
- regset_head tmp_uses, tmp_sets;
- INIT_REG_SET (&tmp_uses);
- INIT_REG_SET (&tmp_sets);
-
- (*current_sched_info->compute_jump_reg_dependencies)
- (insn, &deps->reg_conditional_sets, &tmp_uses, &tmp_sets);
- /* Make latency of jump equal to 0 by using anti-dependence. */
- EXECUTE_IF_SET_IN_REG_SET (&tmp_uses, 0, i, rsi)
- {
- struct deps_reg *reg_last = &deps->reg_last[i];
- add_dependence_list (insn, reg_last->sets, 0, REG_DEP_ANTI);
- add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_ANTI);
- reg_last->uses_length++;
- reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
- }
- IOR_REG_SET (reg_pending_sets, &tmp_sets);
- CLEAR_REG_SET (&tmp_uses);
- CLEAR_REG_SET (&tmp_sets);
+ if (sched_deps_info->compute_jump_reg_dependencies)
+ {
+ regset_head tmp_uses, tmp_sets;
+ INIT_REG_SET (&tmp_uses);
+ INIT_REG_SET (&tmp_sets);
+
+ (*sched_deps_info->compute_jump_reg_dependencies)
+ (insn, &deps->reg_conditional_sets, &tmp_uses, &tmp_sets);
+ /* Make latency of jump equal to 0 by using anti-dependence. */
+ EXECUTE_IF_SET_IN_REG_SET (&tmp_uses, 0, i, rsi)
+ {
+ struct deps_reg *reg_last = &deps->reg_last[i];
+ add_dependence_list (insn, reg_last->sets, 0, REG_DEP_ANTI);
+ add_dependence_list (insn, reg_last->clobbers, 0,
+ REG_DEP_ANTI);
+
+ if (!deps->readonly)
+ {
+ reg_last->uses_length++;
+ reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
+ }
+ }
+ IOR_REG_SET (reg_pending_sets, &tmp_sets);
+
+ CLEAR_REG_SET (&tmp_uses);
+ CLEAR_REG_SET (&tmp_sets);
+ }
/* All memory writes and volatile reads must happen before the
jump. Non-volatile reads must happen before the jump iff
|| (NONJUMP_INSN_P (insn) && control_flow_insn_p (insn)))
reg_pending_barrier = MOVE_BARRIER;
- /* Add register dependencies for insn.
- If the current insn is conditional, we can't free any of the lists. */
- if (sched_get_condition (insn))
+ /* If the current insn is conditional, we can't free any
+ of the lists. */
+ if (sched_has_condition_p (insn))
{
EXECUTE_IF_SET_IN_REG_SET (reg_pending_uses, 0, i, rsi)
- {
- struct deps_reg *reg_last = &deps->reg_last[i];
- add_dependence_list (insn, reg_last->sets, 0, REG_DEP_TRUE);
- add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_TRUE);
- reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
- reg_last->uses_length++;
- }
+ {
+ struct deps_reg *reg_last = &deps->reg_last[i];
+ add_dependence_list (insn, reg_last->sets, 0, REG_DEP_TRUE);
+ add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_TRUE);
+
+ if (!deps->readonly)
+ {
+ reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
+ reg_last->uses_length++;
+ }
+ }
EXECUTE_IF_SET_IN_REG_SET (reg_pending_clobbers, 0, i, rsi)
- {
- struct deps_reg *reg_last = &deps->reg_last[i];
- add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
- add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
- reg_last->clobbers = alloc_INSN_LIST (insn, reg_last->clobbers);
- reg_last->clobbers_length++;
- }
+ {
+ struct deps_reg *reg_last = &deps->reg_last[i];
+ add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
+ add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
+
+ if (!deps->readonly)
+ {
+ reg_last->clobbers = alloc_INSN_LIST (insn, reg_last->clobbers);
+ reg_last->clobbers_length++;
+ }
+ }
EXECUTE_IF_SET_IN_REG_SET (reg_pending_sets, 0, i, rsi)
- {
- struct deps_reg *reg_last = &deps->reg_last[i];
- add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
- add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_OUTPUT);
- add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
- reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
- SET_REGNO_REG_SET (&deps->reg_conditional_sets, i);
- }
+ {
+ struct deps_reg *reg_last = &deps->reg_last[i];
+ add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
+ add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_OUTPUT);
+ add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
+
+ if (!deps->readonly)
+ {
+ reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
+ SET_REGNO_REG_SET (&deps->reg_conditional_sets, i);
+ }
+ }
}
else
{
EXECUTE_IF_SET_IN_REG_SET (reg_pending_uses, 0, i, rsi)
- {
- struct deps_reg *reg_last = &deps->reg_last[i];
- add_dependence_list (insn, reg_last->sets, 0, REG_DEP_TRUE);
- add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_TRUE);
- reg_last->uses_length++;
- reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
- }
+ {
+ struct deps_reg *reg_last = &deps->reg_last[i];
+ add_dependence_list (insn, reg_last->sets, 0, REG_DEP_TRUE);
+ add_dependence_list (insn, reg_last->clobbers, 0, REG_DEP_TRUE);
+
+ if (!deps->readonly)
+ {
+ reg_last->uses_length++;
+ reg_last->uses = alloc_INSN_LIST (insn, reg_last->uses);
+ }
+ }
EXECUTE_IF_SET_IN_REG_SET (reg_pending_clobbers, 0, i, rsi)
- {
- struct deps_reg *reg_last = &deps->reg_last[i];
- if (reg_last->uses_length > MAX_PENDING_LIST_LENGTH
- || reg_last->clobbers_length > MAX_PENDING_LIST_LENGTH)
- {
- add_dependence_list_and_free (insn, ®_last->sets, 0,
- REG_DEP_OUTPUT);
- add_dependence_list_and_free (insn, ®_last->uses, 0,
- REG_DEP_ANTI);
- add_dependence_list_and_free (insn, ®_last->clobbers, 0,
- REG_DEP_OUTPUT);
- reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
- reg_last->clobbers_length = 0;
- reg_last->uses_length = 0;
- }
- else
- {
- add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
- add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
- }
- reg_last->clobbers_length++;
- reg_last->clobbers = alloc_INSN_LIST (insn, reg_last->clobbers);
- }
+ {
+ struct deps_reg *reg_last = &deps->reg_last[i];
+ if (reg_last->uses_length > MAX_PENDING_LIST_LENGTH
+ || reg_last->clobbers_length > MAX_PENDING_LIST_LENGTH)
+ {
+ add_dependence_list_and_free (deps, insn, ®_last->sets, 0,
+ REG_DEP_OUTPUT);
+ add_dependence_list_and_free (deps, insn, ®_last->uses, 0,
+ REG_DEP_ANTI);
+ add_dependence_list_and_free (deps, insn, ®_last->clobbers, 0,
+ REG_DEP_OUTPUT);
+
+ if (!deps->readonly)
+ {
+ reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
+ reg_last->clobbers_length = 0;
+ reg_last->uses_length = 0;
+ }
+ }
+ else
+ {
+ add_dependence_list (insn, reg_last->sets, 0, REG_DEP_OUTPUT);
+ add_dependence_list (insn, reg_last->uses, 0, REG_DEP_ANTI);
+ }
+
+ if (!deps->readonly)
+ {
+ reg_last->clobbers_length++;
+ reg_last->clobbers = alloc_INSN_LIST (insn, reg_last->clobbers);
+ }
+ }
EXECUTE_IF_SET_IN_REG_SET (reg_pending_sets, 0, i, rsi)
- {
- struct deps_reg *reg_last = &deps->reg_last[i];
- add_dependence_list_and_free (insn, ®_last->sets, 0,
- REG_DEP_OUTPUT);
- add_dependence_list_and_free (insn, ®_last->clobbers, 0,
- REG_DEP_OUTPUT);
- add_dependence_list_and_free (insn, ®_last->uses, 0,
- REG_DEP_ANTI);
- reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
- reg_last->uses_length = 0;
- reg_last->clobbers_length = 0;
- CLEAR_REGNO_REG_SET (&deps->reg_conditional_sets, i);
- }
+ {
+ struct deps_reg *reg_last = &deps->reg_last[i];
+ add_dependence_list_and_free (deps, insn, ®_last->sets, 0,
+ REG_DEP_OUTPUT);
+ add_dependence_list_and_free (deps, insn, ®_last->clobbers, 0,
+ REG_DEP_OUTPUT);
+ add_dependence_list_and_free (deps, insn, ®_last->uses, 0,
+ REG_DEP_ANTI);
+
+ if (!deps->readonly)
+ {
+ reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
+ reg_last->uses_length = 0;
+ reg_last->clobbers_length = 0;
+ CLEAR_REGNO_REG_SET (&deps->reg_conditional_sets, i);
+ }
+ }
}
- IOR_REG_SET (&deps->reg_last_in_use, reg_pending_uses);
- IOR_REG_SET (&deps->reg_last_in_use, reg_pending_clobbers);
- IOR_REG_SET (&deps->reg_last_in_use, reg_pending_sets);
+ if (!deps->readonly)
+ {
+ IOR_REG_SET (&deps->reg_last_in_use, reg_pending_uses);
+ IOR_REG_SET (&deps->reg_last_in_use, reg_pending_clobbers);
+ IOR_REG_SET (&deps->reg_last_in_use, reg_pending_sets);
+
+ /* Set up the pending barrier found. */
+ deps->last_reg_pending_barrier = reg_pending_barrier;
+ }
CLEAR_REG_SET (reg_pending_uses);
CLEAR_REG_SET (reg_pending_clobbers);
{
/* In the case of barrier the most added dependencies are not
real, so we use anti-dependence here. */
- if (sched_get_condition (insn))
+ if (sched_has_condition_p (insn))
{
EXECUTE_IF_SET_IN_REG_SET (&deps->reg_last_in_use, 0, i, rsi)
{
EXECUTE_IF_SET_IN_REG_SET (&deps->reg_last_in_use, 0, i, rsi)
{
struct deps_reg *reg_last = &deps->reg_last[i];
- add_dependence_list_and_free (insn, ®_last->uses, 0,
+ add_dependence_list_and_free (deps, insn, ®_last->uses, 0,
REG_DEP_ANTI);
add_dependence_list_and_free
- (insn, ®_last->sets, 0,
+ (deps, insn, ®_last->sets, 0,
reg_pending_barrier == TRUE_BARRIER ? REG_DEP_TRUE : REG_DEP_ANTI);
add_dependence_list_and_free
- (insn, ®_last->clobbers, 0,
+ (deps, insn, ®_last->clobbers, 0,
reg_pending_barrier == TRUE_BARRIER ? REG_DEP_TRUE : REG_DEP_ANTI);
- reg_last->uses_length = 0;
- reg_last->clobbers_length = 0;
- }
- }
- for (i = 0; i < (unsigned)deps->max_reg; i++)
- {
- struct deps_reg *reg_last = &deps->reg_last[i];
- reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
- SET_REGNO_REG_SET (&deps->reg_last_in_use, i);
+ if (!deps->readonly)
+ {
+ reg_last->uses_length = 0;
+ reg_last->clobbers_length = 0;
+ }
+ }
}
- flush_pending_lists (deps, insn, true, true);
- CLEAR_REG_SET (&deps->reg_conditional_sets);
+ if (!deps->readonly)
+ for (i = 0; i < (unsigned)deps->max_reg; i++)
+ {
+ struct deps_reg *reg_last = &deps->reg_last[i];
+ reg_last->sets = alloc_INSN_LIST (insn, reg_last->sets);
+ SET_REGNO_REG_SET (&deps->reg_last_in_use, i);
+ }
+
+ /* Flush pending lists on jumps, but not on speculative checks. */
+ if (JUMP_P (insn) && !(sel_sched_p ()
+ && sel_insn_is_speculation_check (insn)))
+ flush_pending_lists (deps, insn, true, true);
+
+ if (!deps->readonly)
+ CLEAR_REG_SET (&deps->reg_conditional_sets);
reg_pending_barrier = NOT_A_BARRIER;
}
if (src_regno < FIRST_PSEUDO_REGISTER
|| dest_regno < FIRST_PSEUDO_REGISTER)
{
- if (deps->in_post_call_group_p == post_call_initial)
+ if (!deps->readonly
+ && deps->in_post_call_group_p == post_call_initial)
deps->in_post_call_group_p = post_call;
- SCHED_GROUP_P (insn) = 1;
- CANT_MOVE (insn) = 1;
+ if (!sel_sched_p () || sched_emulate_haifa_p)
+ {
+ SCHED_GROUP_P (insn) = 1;
+ CANT_MOVE (insn) = 1;
+ }
}
else
{
end_call_group:
- deps->in_post_call_group_p = not_post_call;
+ if (!deps->readonly)
+ deps->in_post_call_group_p = not_post_call;
}
}
- /* Fixup the dependencies in the sched group. */
- if (SCHED_GROUP_P (insn))
- fixup_sched_groups (insn);
-
if ((current_sched_info->flags & DO_SPECULATION)
&& !sched_insn_is_legitimate_for_speculation_p (insn, 0))
/* INSN has an internal dependency (e.g. r14 = [r14]) and thus cannot
be speculated. */
{
- sd_iterator_def sd_it;
- dep_t dep;
-
- for (sd_it = sd_iterator_start (insn, SD_LIST_SPEC_BACK);
- sd_iterator_cond (&sd_it, &dep);)
- change_spec_dep_to_hard (sd_it);
+ if (sel_sched_p ())
+ sel_mark_hard_insn (insn);
+ else
+ {
+ sd_iterator_def sd_it;
+ dep_t dep;
+
+ for (sd_it = sd_iterator_start (insn, SD_LIST_SPEC_BACK);
+ sd_iterator_cond (&sd_it, &dep);)
+ change_spec_dep_to_hard (sd_it);
+ }
}
}
-/* Analyze every insn between HEAD and TAIL inclusive, creating backward
- dependencies for each insn. */
-
+/* Analyze INSN with DEPS as a context. */
void
-sched_analyze (struct deps *deps, rtx head, rtx tail)
+deps_analyze_insn (struct deps *deps, rtx insn)
{
- rtx insn;
+ if (sched_deps_info->start_insn)
+ sched_deps_info->start_insn (insn);
- if (current_sched_info->use_cselib)
- cselib_init (true);
+ if (NONJUMP_INSN_P (insn) || JUMP_P (insn))
+ {
+ /* Make each JUMP_INSN (but not a speculative check)
+ a scheduling barrier for memory references. */
+ if (!deps->readonly
+ && JUMP_P (insn)
+ && !(sel_sched_p ()
+ && sel_insn_is_speculation_check (insn)))
+ {
+ /* Keep the list a reasonable size. */
+ if (deps->pending_flush_length++ > MAX_PENDING_LIST_LENGTH)
+ flush_pending_lists (deps, insn, true, true);
+ else
+ deps->last_pending_memory_flush
+ = alloc_INSN_LIST (insn, deps->last_pending_memory_flush);
+ }
+
+ sched_analyze_insn (deps, PATTERN (insn), insn);
+ }
+ else if (CALL_P (insn))
+ {
+ int i;
+
+ CANT_MOVE (insn) = 1;
+
+ if (find_reg_note (insn, REG_SETJMP, NULL))
+ {
+ /* This is setjmp. Assume that all registers, not just
+ hard registers, may be clobbered by this call. */
+ reg_pending_barrier = MOVE_BARRIER;
+ }
+ else
+ {
+ for (i = 0; i < FIRST_PSEUDO_REGISTER; i++)
+ /* A call may read and modify global register variables. */
+ if (global_regs[i])
+ {
+ SET_REGNO_REG_SET (reg_pending_sets, i);
+ SET_REGNO_REG_SET (reg_pending_uses, i);
+ }
+ /* Other call-clobbered hard regs may be clobbered.
+ Since we only have a choice between 'might be clobbered'
+ and 'definitely not clobbered', we must include all
+ partly call-clobbered registers here. */
+ else if (HARD_REGNO_CALL_PART_CLOBBERED (i, reg_raw_mode[i])
+ || TEST_HARD_REG_BIT (regs_invalidated_by_call, i))
+ SET_REGNO_REG_SET (reg_pending_clobbers, i);
+ /* We don't know what set of fixed registers might be used
+ by the function, but it is certain that the stack pointer
+ is among them, but be conservative. */
+ else if (fixed_regs[i])
+ SET_REGNO_REG_SET (reg_pending_uses, i);
+ /* The frame pointer is normally not used by the function
+ itself, but by the debugger. */
+ /* ??? MIPS o32 is an exception. It uses the frame pointer
+ in the macro expansion of jal but does not represent this
+ fact in the call_insn rtl. */
+ else if (i == FRAME_POINTER_REGNUM
+ || (i == HARD_FRAME_POINTER_REGNUM
+ && (! reload_completed || frame_pointer_needed)))
+ SET_REGNO_REG_SET (reg_pending_uses, i);
+ }
+
+ /* For each insn which shouldn't cross a call, add a dependence
+ between that insn and this call insn. */
+ add_dependence_list_and_free (deps, insn,
+ &deps->sched_before_next_call, 1,
+ REG_DEP_ANTI);
+
+ sched_analyze_insn (deps, PATTERN (insn), insn);
+
+ /* If CALL would be in a sched group, then this will violate
+ convention that sched group insns have dependencies only on the
+ previous instruction.
+
+ Of course one can say: "Hey! What about head of the sched group?"
+ And I will answer: "Basic principles (one dep per insn) are always
+ the same." */
+ gcc_assert (!SCHED_GROUP_P (insn));
+
+ /* In the absence of interprocedural alias analysis, we must flush
+ all pending reads and writes, and start new dependencies starting
+ from here. But only flush writes for constant calls (which may
+ be passed a pointer to something we haven't written yet). */
+ flush_pending_lists (deps, insn, true, ! RTL_CONST_OR_PURE_CALL_P (insn));
+
+ if (!deps->readonly)
+ {
+ /* Remember the last function call for limiting lifetimes. */
+ free_INSN_LIST_list (&deps->last_function_call);
+ deps->last_function_call = alloc_INSN_LIST (insn, NULL_RTX);
+
+ /* Before reload, begin a post-call group, so as to keep the
+ lifetimes of hard registers correct. */
+ if (! reload_completed)
+ deps->in_post_call_group_p = post_call;
+ }
+ }
+
+ if (sched_deps_info->use_cselib)
+ cselib_process_insn (insn);
+
+ /* EH_REGION insn notes can not appear until well after we complete
+ scheduling. */
+ if (NOTE_P (insn))
+ gcc_assert (NOTE_KIND (insn) != NOTE_INSN_EH_REGION_BEG
+ && NOTE_KIND (insn) != NOTE_INSN_EH_REGION_END);
+
+ if (sched_deps_info->finish_insn)
+ sched_deps_info->finish_insn ();
+
+ /* Fixup the dependencies in the sched group. */
+ if ((NONJUMP_INSN_P (insn) || JUMP_P (insn))
+ && SCHED_GROUP_P (insn) && !sel_sched_p ())
+ fixup_sched_groups (insn);
+}
+
+/* Initialize DEPS for the new block beginning with HEAD. */
+void
+deps_start_bb (struct deps *deps, rtx head)
+{
+ gcc_assert (!deps->readonly);
/* Before reload, if the previous block ended in a call, show that
we are inside a post-call group, so as to keep the lifetimes of
hard registers correct. */
if (! reload_completed && !LABEL_P (head))
{
- insn = prev_nonnote_insn (head);
+ rtx insn = prev_nonnote_insn (head);
+
if (insn && CALL_P (insn))
deps->in_post_call_group_p = post_call_initial;
}
+}
+
+/* Analyze every insn between HEAD and TAIL inclusive, creating backward
+ dependencies for each insn. */
+void
+sched_analyze (struct deps *deps, rtx head, rtx tail)
+{
+ rtx insn;
+
+ if (sched_deps_info->use_cselib)
+ cselib_init (true);
+
+ deps_start_bb (deps, head);
+
for (insn = head;; insn = NEXT_INSN (insn))
{
+
if (INSN_P (insn))
{
/* And initialize deps_lists. */
sd_init_insn (insn);
}
- if (NONJUMP_INSN_P (insn) || JUMP_P (insn))
- {
- /* Make each JUMP_INSN a scheduling barrier for memory
- references. */
- if (JUMP_P (insn))
- {
- /* Keep the list a reasonable size. */
- if (deps->pending_flush_length++ > MAX_PENDING_LIST_LENGTH)
- flush_pending_lists (deps, insn, true, true);
- else
- deps->last_pending_memory_flush
- = alloc_INSN_LIST (insn, deps->last_pending_memory_flush);
- }
- sched_analyze_insn (deps, PATTERN (insn), insn);
- }
- else if (CALL_P (insn))
- {
- int i;
-
- CANT_MOVE (insn) = 1;
-
- if (find_reg_note (insn, REG_SETJMP, NULL))
- {
- /* This is setjmp. Assume that all registers, not just
- hard registers, may be clobbered by this call. */
- reg_pending_barrier = MOVE_BARRIER;
- }
- else
- {
- for (i = 0; i < FIRST_PSEUDO_REGISTER; i++)
- /* A call may read and modify global register variables. */
- if (global_regs[i])
- {
- SET_REGNO_REG_SET (reg_pending_sets, i);
- SET_REGNO_REG_SET (reg_pending_uses, i);
- }
- /* Other call-clobbered hard regs may be clobbered.
- Since we only have a choice between 'might be clobbered'
- and 'definitely not clobbered', we must include all
- partly call-clobbered registers here. */
- else if (HARD_REGNO_CALL_PART_CLOBBERED (i, reg_raw_mode[i])
- || TEST_HARD_REG_BIT (regs_invalidated_by_call, i))
- SET_REGNO_REG_SET (reg_pending_clobbers, i);
- /* We don't know what set of fixed registers might be used
- by the function, but it is certain that the stack pointer
- is among them, but be conservative. */
- else if (fixed_regs[i])
- SET_REGNO_REG_SET (reg_pending_uses, i);
- /* The frame pointer is normally not used by the function
- itself, but by the debugger. */
- /* ??? MIPS o32 is an exception. It uses the frame pointer
- in the macro expansion of jal but does not represent this
- fact in the call_insn rtl. */
- else if (i == FRAME_POINTER_REGNUM
- || (i == HARD_FRAME_POINTER_REGNUM
- && (! reload_completed || frame_pointer_needed)))
- SET_REGNO_REG_SET (reg_pending_uses, i);
- }
-
- /* For each insn which shouldn't cross a call, add a dependence
- between that insn and this call insn. */
- add_dependence_list_and_free (insn, &deps->sched_before_next_call, 1,
- REG_DEP_ANTI);
-
- sched_analyze_insn (deps, PATTERN (insn), insn);
-
- /* In the absence of interprocedural alias analysis, we must flush
- all pending reads and writes, and start new dependencies starting
- from here. But only flush writes for constant calls (which may
- be passed a pointer to something we haven't written yet). */
- flush_pending_lists (deps, insn, true,
- ! RTL_CONST_OR_PURE_CALL_P (insn));
-
- /* Remember the last function call for limiting lifetimes. */
- free_INSN_LIST_list (&deps->last_function_call);
- deps->last_function_call = alloc_INSN_LIST (insn, NULL_RTX);
-
- /* Before reload, begin a post-call group, so as to keep the
- lifetimes of hard registers correct. */
- if (! reload_completed)
- deps->in_post_call_group_p = post_call;
- }
-
- /* EH_REGION insn notes can not appear until well after we complete
- scheduling. */
- if (NOTE_P (insn))
- gcc_assert (NOTE_KIND (insn) != NOTE_INSN_EH_REGION_BEG
- && NOTE_KIND (insn) != NOTE_INSN_EH_REGION_END);
-
- if (current_sched_info->use_cselib)
- cselib_process_insn (insn);
+ deps_analyze_insn (deps, insn);
if (insn == tail)
{
- if (current_sched_info->use_cselib)
+ if (sched_deps_info->use_cselib)
cselib_finish ();
return;
}
deps->last_function_call = 0;
deps->sched_before_next_call = 0;
deps->in_post_call_group_p = not_post_call;
+ deps->last_reg_pending_barrier = NOT_A_BARRIER;
+ deps->readonly = 0;
}
/* Free insn lists found in DEPS. */
CLEAR_REG_SET (&deps->reg_conditional_sets);
free (deps->reg_last);
+ deps->reg_last = NULL;
+
+ deps = NULL;
}
-/* If it is profitable to use them, initialize caches for tracking
- dependency information. LUID is the number of insns to be scheduled,
- it is used in the estimate of profitability. */
+/* Remove INSN from dependence contexts DEPS. Caution: reg_conditional_sets
+ is not handled. */
+void
+remove_from_deps (struct deps *deps, rtx insn)
+{
+ int removed;
+ unsigned i;
+ reg_set_iterator rsi;
+
+ removed = remove_from_both_dependence_lists (insn, &deps->pending_read_insns,
+ &deps->pending_read_mems);
+ deps->pending_read_list_length -= removed;
+ removed = remove_from_both_dependence_lists (insn, &deps->pending_write_insns,
+ &deps->pending_write_mems);
+ deps->pending_write_list_length -= removed;
+ removed = remove_from_dependence_list (insn, &deps->last_pending_memory_flush);
+ deps->pending_flush_length -= removed;
+ EXECUTE_IF_SET_IN_REG_SET (&deps->reg_last_in_use, 0, i, rsi)
+ {
+ struct deps_reg *reg_last = &deps->reg_last[i];
+ if (reg_last->uses)
+ remove_from_dependence_list (insn, ®_last->uses);
+ if (reg_last->sets)
+ remove_from_dependence_list (insn, ®_last->sets);
+ if (reg_last->clobbers)
+ remove_from_dependence_list (insn, ®_last->clobbers);
+ if (!reg_last->uses && !reg_last->sets && !reg_last->clobbers)
+ CLEAR_REGNO_REG_SET (&deps->reg_last_in_use, i);
+ }
+
+ if (CALL_P (insn))
+ remove_from_dependence_list (insn, &deps->last_function_call);
+ remove_from_dependence_list (insn, &deps->sched_before_next_call);
+}
+
+/* Init deps data vector. */
+static void
+init_deps_data_vector (void)
+{
+ int reserve = (sched_max_luid + 1
+ - VEC_length (haifa_deps_insn_data_def, h_d_i_d));
+ if (reserve > 0
+ && ! VEC_space (haifa_deps_insn_data_def, h_d_i_d, reserve))
+ VEC_safe_grow_cleared (haifa_deps_insn_data_def, heap, h_d_i_d,
+ 3 * sched_max_luid / 2);
+}
+
+/* If it is profitable to use them, initialize or extend (depending on
+ GLOBAL_P) dependency data. */
void
-init_dependency_caches (int luid)
+sched_deps_init (bool global_p)
{
/* Average number of insns in the basic block.
'+ 1' is used to make it nonzero. */
- int insns_in_block = luid / n_basic_blocks + 1;
+ int insns_in_block = sched_max_luid / n_basic_blocks + 1;
- /* ?!? We could save some memory by computing a per-region luid mapping
- which could reduce both the number of vectors in the cache and the size
- of each vector. Instead we just avoid the cache entirely unless the
- average number of instructions in a basic block is very high. See
- the comment before the declaration of true_dependency_cache for
- what we consider "very high". */
- if (insns_in_block > 100 * 5)
- {
+ init_deps_data_vector ();
+
+ /* We use another caching mechanism for selective scheduling, so
+ we don't use this one. */
+ if (!sel_sched_p () && global_p && insns_in_block > 100 * 5)
+ {
+ /* ?!? We could save some memory by computing a per-region luid mapping
+ which could reduce both the number of vectors in the cache and the
+ size of each vector. Instead we just avoid the cache entirely unless
+ the average number of instructions in a basic block is very high. See
+ the comment before the declaration of true_dependency_cache for
+ what we consider "very high". */
cache_size = 0;
- extend_dependency_caches (luid, true);
+ extend_dependency_caches (sched_max_luid, true);
}
- dl_pool = create_alloc_pool ("deps_list", sizeof (struct _deps_list),
- /* Allocate lists for one block at a time. */
- insns_in_block);
-
- dn_pool = create_alloc_pool ("dep_node", sizeof (struct _dep_node),
- /* Allocate nodes for one block at a time.
- We assume that average insn has
- 5 producers. */
- 5 * insns_in_block);
+ if (global_p)
+ {
+ dl_pool = create_alloc_pool ("deps_list", sizeof (struct _deps_list),
+ /* Allocate lists for one block at a time. */
+ insns_in_block);
+ dn_pool = create_alloc_pool ("dep_node", sizeof (struct _dep_node),
+ /* Allocate nodes for one block at a time.
+ We assume that average insn has
+ 5 producers. */
+ 5 * insns_in_block);
+ }
}
+
/* Create or extend (depending on CREATE_P) dependency caches to
size N. */
void
}
}
-/* Free the caches allocated in init_dependency_caches. */
-
+/* Finalize dependency information for the whole function. */
void
-free_dependency_caches (void)
+sched_deps_finish (void)
{
gcc_assert (deps_pools_are_empty_p ());
free_alloc_pool_if_empty (&dn_pool);
free_alloc_pool_if_empty (&dl_pool);
gcc_assert (dn_pool == NULL && dl_pool == NULL);
+ VEC_free (haifa_deps_insn_data_def, heap, h_d_i_d);
+ cache_size = 0;
+
if (true_dependency_cache)
{
int i;
bitmap_clear (&output_dependency_cache[i]);
bitmap_clear (&anti_dependency_cache[i]);
- if (current_sched_info->flags & DO_SPECULATION)
+ if (sched_deps_info->generate_spec_deps)
bitmap_clear (&spec_dependency_cache[i]);
}
free (true_dependency_cache);
free (anti_dependency_cache);
anti_dependency_cache = NULL;
- if (current_sched_info->flags & DO_SPECULATION)
+ if (sched_deps_info->generate_spec_deps)
{
free (spec_dependency_cache);
spec_dependency_cache = NULL;
}
+
}
}
reg_pending_clobbers = ALLOC_REG_SET (®_obstack);
reg_pending_uses = ALLOC_REG_SET (®_obstack);
reg_pending_barrier = NOT_A_BARRIER;
+
+ if (!sel_sched_p () || sched_emulate_haifa_p)
+ {
+ sched_deps_info->start_insn = haifa_start_insn;
+ sched_deps_info->finish_insn = haifa_finish_insn;
+
+ sched_deps_info->note_reg_set = haifa_note_reg_set;
+ sched_deps_info->note_reg_clobber = haifa_note_reg_clobber;
+ sched_deps_info->note_reg_use = haifa_note_reg_use;
+
+ sched_deps_info->note_mem_dep = haifa_note_mem_dep;
+ sched_deps_info->note_dep = haifa_note_dep;
+ }
}
/* Free everything used by the dependency analysis code. */
}
/* Estimate the weakness of dependence between MEM1 and MEM2. */
-static dw_t
+dw_t
estimate_dep_weak (rtx mem1, rtx mem2)
{
rtx r1, r2;
void
add_dependence (rtx insn, rtx elem, enum reg_note dep_type)
{
- dep_def _dep, *dep = &_dep;
+ ds_t ds;
+ bool internal;
- init_dep (dep, elem, insn, dep_type);
- maybe_add_or_update_dep_1 (dep, false, NULL_RTX, NULL_RTX);
+ if (dep_type == REG_DEP_TRUE)
+ ds = DEP_TRUE;
+ else if (dep_type == REG_DEP_OUTPUT)
+ ds = DEP_OUTPUT;
+ else
+ {
+ gcc_assert (dep_type == REG_DEP_ANTI);
+ ds = DEP_ANTI;
+ }
+
+ /* When add_dependence is called from inside sched-deps.c, we expect
+ cur_insn to be non-null. */
+ internal = cur_insn != NULL;
+ if (internal)
+ gcc_assert (insn == cur_insn);
+ else
+ cur_insn = insn;
+
+ note_dep (elem, ds);
+ if (!internal)
+ cur_insn = NULL;
}
/* Return weakness of speculative type TYPE in the dep_status DS. */
-static dw_t
+dw_t
get_dep_weak_1 (ds_t ds, ds_t type)
{
ds = ds & type;
+
switch (type)
{
case BEGIN_DATA: ds >>= BEGIN_DATA_BITS_OFFSET; break;
return (dw_t) ds;
}
-/* Return weakness of speculative type TYPE in the dep_status DS. */
dw_t
get_dep_weak (ds_t ds, ds_t type)
{
dw_t dw = get_dep_weak_1 (ds, type);
gcc_assert (MIN_DEP_WEAK <= dw && dw <= MAX_DEP_WEAK);
-
return dw;
}
return ds;
}
-/* Return the join of two dep_statuses DS1 and DS2. */
-ds_t
-ds_merge (ds_t ds1, ds_t ds2)
+/* Return the join of two dep_statuses DS1 and DS2.
+ If MAX_P is true then choose the greater probability,
+ otherwise multiply probabilities.
+ This function assumes that both DS1 and DS2 contain speculative bits. */
+static ds_t
+ds_merge_1 (ds_t ds1, ds_t ds2, bool max_p)
{
ds_t ds, t;
ds |= ds2 & t;
else if ((ds1 & t) && (ds2 & t))
{
+ dw_t dw1 = get_dep_weak (ds1, t);
+ dw_t dw2 = get_dep_weak (ds2, t);
ds_t dw;
- dw = ((ds_t) get_dep_weak (ds1, t)) * ((ds_t) get_dep_weak (ds2, t));
- dw /= MAX_DEP_WEAK;
- if (dw < MIN_DEP_WEAK)
- dw = MIN_DEP_WEAK;
+ if (!max_p)
+ {
+ dw = ((ds_t) dw1) * ((ds_t) dw2);
+ dw /= MAX_DEP_WEAK;
+ if (dw < MIN_DEP_WEAK)
+ dw = MIN_DEP_WEAK;
+ }
+ else
+ {
+ if (dw1 >= dw2)
+ dw = dw1;
+ else
+ dw = dw2;
+ }
ds = set_dep_weak (ds, t, (dw_t) dw);
}
return ds;
}
+/* Return the join of two dep_statuses DS1 and DS2.
+ This function assumes that both DS1 and DS2 contain speculative bits. */
+ds_t
+ds_merge (ds_t ds1, ds_t ds2)
+{
+ return ds_merge_1 (ds1, ds2, false);
+}
+
+/* Return the join of two dep_statuses DS1 and DS2. */
+ds_t
+ds_full_merge (ds_t ds, ds_t ds2, rtx mem1, rtx mem2)
+{
+ ds_t new_status = ds | ds2;
+
+ if (new_status & SPECULATIVE)
+ {
+ if ((ds && !(ds & SPECULATIVE))
+ || (ds2 && !(ds2 & SPECULATIVE)))
+ /* Then this dep can't be speculative. */
+ new_status &= ~SPECULATIVE;
+ else
+ {
+ /* Both are speculative. Merging probabilities. */
+ if (mem1)
+ {
+ dw_t dw;
+
+ dw = estimate_dep_weak (mem1, mem2);
+ ds = set_dep_weak (ds, BEGIN_DATA, dw);
+ }
+
+ if (!ds)
+ new_status = ds2;
+ else if (!ds2)
+ new_status = ds;
+ else
+ new_status = ds_merge (ds2, ds);
+ }
+ }
+
+ return new_status;
+}
+
+/* Return the join of DS1 and DS2. Use maximum instead of multiplying
+ probabilities. */
+ds_t
+ds_max_merge (ds_t ds1, ds_t ds2)
+{
+ if (ds1 == 0 && ds2 == 0)
+ return 0;
+
+ if (ds1 == 0 && ds2 != 0)
+ return ds2;
+
+ if (ds1 != 0 && ds2 == 0)
+ return ds1;
+
+ return ds_merge_1 (ds1, ds2, true);
+}
+
+/* Return the probability of speculation success for the speculation
+ status DS. */
+dw_t
+ds_weak (ds_t ds)
+{
+ ds_t res = 1, dt;
+ int n = 0;
+
+ dt = FIRST_SPEC_TYPE;
+ do
+ {
+ if (ds & dt)
+ {
+ res *= (ds_t) get_dep_weak (ds, dt);
+ n++;
+ }
+
+ if (dt == LAST_SPEC_TYPE)
+ break;
+ dt <<= SPEC_TYPE_SHIFT;
+ }
+ while (1);
+
+ gcc_assert (n);
+ while (--n)
+ res /= MAX_DEP_WEAK;
+
+ if (res < MIN_DEP_WEAK)
+ res = MIN_DEP_WEAK;
+
+ gcc_assert (res <= MAX_DEP_WEAK);
+
+ return (dw_t) res;
+}
+
+/* Return a dep status that contains all speculation types of DS. */
+ds_t
+ds_get_speculation_types (ds_t ds)
+{
+ if (ds & BEGIN_DATA)
+ ds |= BEGIN_DATA;
+ if (ds & BE_IN_DATA)
+ ds |= BE_IN_DATA;
+ if (ds & BEGIN_CONTROL)
+ ds |= BEGIN_CONTROL;
+ if (ds & BE_IN_CONTROL)
+ ds |= BE_IN_CONTROL;
+
+ return ds & SPECULATIVE;
+}
+
+/* Return a dep status that contains maximal weakness for each speculation
+ type present in DS. */
+ds_t
+ds_get_max_dep_weak (ds_t ds)
+{
+ if (ds & BEGIN_DATA)
+ ds = set_dep_weak (ds, BEGIN_DATA, MAX_DEP_WEAK);
+ if (ds & BE_IN_DATA)
+ ds = set_dep_weak (ds, BE_IN_DATA, MAX_DEP_WEAK);
+ if (ds & BEGIN_CONTROL)
+ ds = set_dep_weak (ds, BEGIN_CONTROL, MAX_DEP_WEAK);
+ if (ds & BE_IN_CONTROL)
+ ds = set_dep_weak (ds, BE_IN_CONTROL, MAX_DEP_WEAK);
+
+ return ds;
+}
+
/* Dump information about the dependence status S. */
static void
dump_ds (FILE *f, ds_t s)
/* Check that dependence status is set correctly when speculation is not
supported. */
- if (!(current_sched_info->flags & DO_SPECULATION))
+ if (!sched_deps_info->generate_spec_deps)
gcc_assert (!(ds & SPECULATIVE));
else if (ds & SPECULATIVE)
{
\f
#ifdef INSN_SCHEDULING
-/* The number of insns scheduled so far. */
-static int sched_n_insns;
-
/* The number of insns to be scheduled in total. */
-static int n_insns;
+static int rgn_n_insns;
+
+/* The number of insns scheduled so far. */
+static int sched_rgn_n_insns;
/* Set of blocks, that already have their dependencies calculated. */
static bitmap_head dont_calc_deps;
static void init_ready_list (void);
static void begin_schedule_ready (rtx, rtx);
static int schedule_more_p (void);
-static const char *ebb_print_insn (rtx, int);
+static const char *ebb_print_insn (const_rtx, int);
static int rank (rtx, rtx);
-static int contributes_to_priority (rtx, rtx);
-static void compute_jump_reg_dependencies (rtx, regset, regset, regset);
+static int ebb_contributes_to_priority (rtx, rtx);
static basic_block earliest_block_with_similiar_load (basic_block, rtx);
static void add_deps_for_risky_insns (rtx, rtx);
static basic_block schedule_ebb (rtx, rtx);
+static void debug_ebb_dependencies (rtx, rtx);
-static void add_remove_insn (rtx, int);
-static void add_block1 (basic_block, basic_block);
+static void ebb_add_remove_insn (rtx, int);
+static void ebb_add_block (basic_block, basic_block);
static basic_block advance_target_bb (basic_block, rtx);
-static void fix_recovery_cfg (int, int, int);
+static void ebb_fix_recovery_cfg (int, int, int);
/* Return nonzero if there are more insns that should be scheduled. */
static int
schedule_more_p (void)
{
- return sched_n_insns < n_insns;
+ return sched_rgn_n_insns < rgn_n_insns;
}
/* Print dependency information about ebb between HEAD and TAIL. */
rtx next_tail = current_sched_info->next_tail;
rtx insn;
- sched_n_insns = 0;
+ sched_rgn_n_insns = 0;
/* Print debugging information. */
if (sched_verbose >= 5)
n++;
}
- gcc_assert (n == n_insns);
+ gcc_assert (n == rgn_n_insns);
}
/* INSN is being scheduled after LAST. Update counters. */
static void
begin_schedule_ready (rtx insn, rtx last)
{
- sched_n_insns++;
+ sched_rgn_n_insns++;
if (BLOCK_FOR_INSN (insn) == last_bb
/* INSN is a jump in the last block, ... */
current_sched_info->next_tail = NEXT_INSN (BB_END (bb));
gcc_assert (current_sched_info->next_tail);
- add_block (bb, last_bb);
+ /* Append new basic block to the end of the ebb. */
+ sched_init_only_bb (bb, last_bb);
gcc_assert (last_bb == bb);
}
}
to be formatted so that multiple output lines will line up nicely. */
static const char *
-ebb_print_insn (rtx insn, int aligned ATTRIBUTE_UNUSED)
+ebb_print_insn (const_rtx insn, int aligned ATTRIBUTE_UNUSED)
{
static char tmp[80];
- sprintf (tmp, "%4d", INSN_UID (insn));
+ /* '+' before insn means it is a new cycle start. */
+ if (GET_MODE (insn) == TImode)
+ sprintf (tmp, "+ %4d", INSN_UID (insn));
+ else
+ sprintf (tmp, " %4d", INSN_UID (insn));
+
return tmp;
}
calculations. */
static int
-contributes_to_priority (rtx next ATTRIBUTE_UNUSED,
- rtx insn ATTRIBUTE_UNUSED)
+ebb_contributes_to_priority (rtx next ATTRIBUTE_UNUSED,
+ rtx insn ATTRIBUTE_UNUSED)
{
return 1;
}
must be considered as used by this jump in USED and that of
registers that must be considered as set in SET. */
-static void
-compute_jump_reg_dependencies (rtx insn, regset cond_set, regset used,
- regset set)
+void
+ebb_compute_jump_reg_dependencies (rtx insn, regset cond_set, regset used,
+ regset set)
{
basic_block b = BLOCK_FOR_INSN (insn);
edge e;
/* Used in schedule_insns to initialize current_sched_info for scheduling
regions (or single basic blocks). */
-static struct sched_info ebb_sched_info =
+static struct common_sched_info_def ebb_common_sched_info;
+
+static struct sched_deps_info_def ebb_sched_deps_info =
+ {
+ ebb_compute_jump_reg_dependencies,
+ NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,
+ NULL,
+ 1, 0, 0
+ };
+
+static struct haifa_sched_info ebb_sched_info =
{
init_ready_list,
NULL,
NULL,
rank,
ebb_print_insn,
- contributes_to_priority,
- compute_jump_reg_dependencies,
+ ebb_contributes_to_priority,
NULL, NULL,
NULL, NULL,
- 0, 1, 0,
+ 1, 0,
- add_remove_insn,
+ ebb_add_remove_insn,
begin_schedule_ready,
- add_block1,
advance_target_bb,
- fix_recovery_cfg,
SCHED_EBB
/* We can create new blocks in begin_schedule_ready (). */
| NEW_BBS
/* Set priorities. */
current_sched_info->sched_max_insns_priority = 0;
- n_insns = set_priorities (head, tail);
+ rgn_n_insns = set_priorities (head, tail);
current_sched_info->sched_max_insns_priority++;
current_sched_info->prev_head = PREV_INSN (head);
current_sched_info->next_tail = NEXT_INSN (tail);
- /* rm_other_notes only removes notes which are _inside_ the
- block---that is, it won't remove notes before the first real insn
- or after the last real insn of the block. So if the first insn
- has a REG_SAVE_NOTE which would otherwise be emitted before the
- insn, it is redundant with the note before the start of the
- block, and so we have to take it out. */
- if (INSN_P (head))
- {
- rtx note;
-
- for (note = REG_NOTES (head); note; note = XEXP (note, 1))
- if (REG_NOTE_KIND (note) == REG_SAVE_NOTE)
- remove_note (head, note);
- }
-
- /* Remove remaining note insns from the block, save them in
- note_list. These notes are restored at the end of
- schedule_block (). */
- rm_other_notes (head, tail);
+ remove_notes (head, tail);
unlink_bb_notes (first_bb, last_bb);
- current_sched_info->queue_must_finish_empty = 1;
-
target_bb = first_bb;
- schedule_block (&target_bb, n_insns);
+
+ /* Make ready list big enough to hold all the instructions from the ebb. */
+ sched_extend_ready_list (rgn_n_insns);
+ schedule_block (&target_bb);
+ /* Free ready list. */
+ sched_finish_ready_list ();
/* We might pack all instructions into fewer blocks,
so we may made some of them empty. Can't assert (b == last_bb). */
/* Sanity check: verify that all region insns were scheduled. */
- gcc_assert (sched_n_insns == n_insns);
+ gcc_assert (sched_rgn_n_insns == rgn_n_insns);
/* Free dependencies. */
sched_free_deps (current_sched_info->head, current_sched_info->tail, true);
if (n_basic_blocks == NUM_FIXED_BLOCKS)
return;
- /* We need current_sched_info in init_dependency_caches, which is
- invoked via sched_init. */
- current_sched_info = &ebb_sched_info;
+ /* Setup infos. */
+ {
+ memcpy (&ebb_common_sched_info, &haifa_common_sched_info,
+ sizeof (ebb_common_sched_info));
+
+ ebb_common_sched_info.fix_recovery_cfg = ebb_fix_recovery_cfg;
+ ebb_common_sched_info.add_block = ebb_add_block;
+ ebb_common_sched_info.sched_pass_id = SCHED_EBB_PASS;
+
+ common_sched_info = &ebb_common_sched_info;
+ sched_deps_info = &ebb_sched_deps_info;
+ current_sched_info = &ebb_sched_info;
+ }
- df_set_flags (DF_LR_RUN_DCE);
- df_note_add_problem ();
- df_analyze ();
- df_clear_flags (DF_LR_RUN_DCE);
- regstat_compute_calls_crossed ();
- sched_init ();
+ haifa_sched_init ();
compute_bb_for_insn ();
if (reload_completed)
reposition_prologue_and_epilogue_notes ();
- sched_finish ();
- regstat_free_calls_crossed ();
+ haifa_sched_finish ();
}
/* INSN has been added to/removed from current ebb. */
static void
-add_remove_insn (rtx insn ATTRIBUTE_UNUSED, int remove_p)
+ebb_add_remove_insn (rtx insn ATTRIBUTE_UNUSED, int remove_p)
{
if (!remove_p)
- n_insns++;
+ rgn_n_insns++;
else
- n_insns--;
+ rgn_n_insns--;
}
/* BB was added to ebb after AFTER. */
static void
-add_block1 (basic_block bb, basic_block after)
+ebb_add_block (basic_block bb, basic_block after)
{
/* Recovery blocks are always bounded by BARRIERS,
therefore, they always form single block EBB,
For parameter meaning please refer to
sched-int.h: struct sched_info: fix_recovery_cfg. */
static void
-fix_recovery_cfg (int bbi ATTRIBUTE_UNUSED, int jump_bbi, int jump_bb_nexti)
+ebb_fix_recovery_cfg (int bbi ATTRIBUTE_UNUSED, int jump_bbi,
+ int jump_bb_nexti)
{
gcc_assert (last_bb->index != bbi);
/* For state_t. */
#include "insn-attr.h"
-/* For regset_head. */
-#include "basic-block.h"
-/* For reg_note. */
-#include "rtl.h"
#include "df.h"
+#include "basic-block.h"
+
+/* For VEC (int, heap). */
+#include "vecprim.h"
+
+/* Identificator of a scheduler pass. */
+enum sched_pass_id_t { SCHED_PASS_UNKNOWN, SCHED_RGN_PASS, SCHED_EBB_PASS,
+ SCHED_SMS_PASS, SCHED_SEL_PASS };
+
+typedef VEC (basic_block, heap) *bb_vec_t;
+typedef VEC (rtx, heap) *insn_vec_t;
+typedef VEC(rtx, heap) *rtx_vec_t;
+
+struct sched_scan_info_def
+{
+ /* This hook notifies scheduler frontend to extend its internal per basic
+ block data structures. This hook should be called once before a series of
+ calls to bb_init (). */
+ void (*extend_bb) (void);
+
+ /* This hook makes scheduler frontend to initialize its internal data
+ structures for the passed basic block. */
+ void (*init_bb) (basic_block);
+
+ /* This hook notifies scheduler frontend to extend its internal per insn data
+ structures. This hook should be called once before a series of calls to
+ insn_init (). */
+ void (*extend_insn) (void);
+
+ /* This hook makes scheduler frontend to initialize its internal data
+ structures for the passed insn. */
+ void (*init_insn) (rtx);
+};
+
+extern const struct sched_scan_info_def *sched_scan_info;
+
+extern void sched_scan (const struct sched_scan_info_def *,
+ bb_vec_t, basic_block, insn_vec_t, rtx);
+
+extern void sched_init_bbs (void);
+
+extern void sched_init_luids (bb_vec_t, basic_block, insn_vec_t, rtx);
+extern void sched_finish_luids (void);
+
+extern void sched_extend_target (void);
+
+extern void haifa_init_h_i_d (bb_vec_t, basic_block, insn_vec_t, rtx);
+extern void haifa_finish_h_i_d (void);
+
+/* Hooks that are common to all the schedulers. */
+struct common_sched_info_def
+{
+ /* Called after blocks were rearranged due to movement of jump instruction.
+ The first parameter - index of basic block, in which jump currently is.
+ The second parameter - index of basic block, in which jump used
+ to be.
+ The third parameter - index of basic block, that follows the second
+ parameter. */
+ void (*fix_recovery_cfg) (int, int, int);
+
+ /* Called to notify frontend, that new basic block is being added.
+ The first parameter - new basic block.
+ The second parameter - block, after which new basic block is being added,
+ or EXIT_BLOCK_PTR, if recovery block is being added,
+ or NULL, if standalone block is being added. */
+ void (*add_block) (basic_block, basic_block);
+
+ /* Estimate number of insns in the basic block. */
+ int (*estimate_number_of_insns) (basic_block);
+
+ /* Given a non-insn (!INSN_P (x)) return
+ -1 - if this rtx don't need a luid.
+ 0 - if it should have the same luid as the previous insn.
+ 1 - if it needs a separate luid. */
+ int (*luid_for_non_insn) (rtx);
+
+ /* Scheduler pass identifier. It is preferably used in assertions. */
+ enum sched_pass_id_t sched_pass_id;
+};
+
+extern struct common_sched_info_def *common_sched_info;
+
+extern const struct common_sched_info_def haifa_common_sched_info;
+
+/* Return true if selective scheduling pass is working. */
+static inline bool
+sel_sched_p (void)
+{
+ return common_sched_info->sched_pass_id == SCHED_SEL_PASS;
+}
+
+/* Returns maximum priority that an insn was assigned to. */
+extern int get_rgn_sched_max_insns_priority (void);
+
+/* Increases effective priority for INSN by AMOUNT. */
+extern void sel_add_to_insn_priority (rtx, int);
+
+/* True if during selective scheduling we need to emulate some of haifa
+ scheduler behaviour. */
+extern int sched_emulate_haifa_p;
+
+/* Mapping from INSN_UID to INSN_LUID. In the end all other per insn data
+ structures should be indexed by luid. */
+extern VEC (int, heap) *sched_luids;
+#define INSN_LUID(INSN) (VEC_index (int, sched_luids, INSN_UID (INSN)))
+#define LUID_BY_UID(UID) (VEC_index (int, sched_luids, UID))
+
+#define SET_INSN_LUID(INSN, LUID) \
+(VEC_replace (int, sched_luids, INSN_UID (INSN), (LUID)))
+
+/* The highest INSN_LUID. */
+extern int sched_max_luid;
+
+extern int insn_luid (rtx);
+
+/* This list holds ripped off notes from the current block. These notes will
+ be attached to the beginning of the block when its scheduling is
+ finished. */
+extern rtx note_list;
+
+extern void remove_notes (rtx, rtx);
+extern rtx restore_other_notes (rtx, basic_block);
+extern void sched_insns_init (rtx);
+extern void sched_insns_finish (void);
+
+extern void *xrecalloc (void *, size_t, size_t, size_t);
+extern rtx bb_note (basic_block);
+
+extern void reemit_notes (rtx);
+
+/* Functions in sched-vis.c. */
+extern void print_insn (char *, const_rtx, int);
+extern void print_pattern (char *, const_rtx, int);
+extern void print_value (char *, const_rtx, int);
+
+/* Functions in haifa-sched.c. */
+extern int haifa_classify_insn (const_rtx);
+
+/* Functions in sel-sched-ir.c. */
+extern void sel_find_rgns (void);
+extern void sel_mark_hard_insn (rtx);
+
+extern size_t dfa_state_size;
+
+extern void advance_state (state_t);
+
+extern void setup_sched_dump (void);
+extern void sched_init (void);
+extern void sched_finish (void);
+
+extern bool sel_insn_is_speculation_check (rtx);
+
+/* Describe the ready list of the scheduler.
+ VEC holds space enough for all insns in the current region. VECLEN
+ says how many exactly.
+ FIRST is the index of the element with the highest priority; i.e. the
+ last one in the ready list, since elements are ordered by ascending
+ priority.
+ N_READY determines how many insns are on the ready list. */
+struct ready_list
+{
+ rtx *vec;
+ int veclen;
+ int first;
+ int n_ready;
+};
+
+extern char *ready_try;
+extern struct ready_list ready;
+
+extern int max_issue (struct ready_list *, int, state_t, int *);
+
+extern void ebb_compute_jump_reg_dependencies (rtx, regset, regset, regset);
+
+extern edge find_fallthru_edge (basic_block);
+
+extern void (* sched_init_only_bb) (basic_block, basic_block);
+extern basic_block (* sched_split_block) (basic_block, rtx);
+extern basic_block sched_split_block_1 (basic_block, rtx);
+extern basic_block (* sched_create_empty_bb) (basic_block);
+extern basic_block sched_create_empty_bb_1 (basic_block);
+
+extern basic_block sched_create_recovery_block (basic_block *);
+extern void sched_create_recovery_edges (basic_block, basic_block,
+ basic_block);
/* Pointer to data describing the current DFA state. */
extern state_t curr_state;
-/* Forward declaration. */
-struct ready_list;
-
/* Type to represent status of a dependence. */
typedef int ds_t;
#define DEP_NODE_DEP(N) (&(N)->dep)
#define DEP_NODE_FORW(N) (&(N)->forw)
+/* The following enumeration values tell us what dependencies we
+ should use to implement the barrier. We use true-dependencies for
+ TRUE_BARRIER and anti-dependencies for MOVE_BARRIER. */
+enum reg_pending_barrier_mode
+{
+ NOT_A_BARRIER = 0,
+ MOVE_BARRIER,
+ TRUE_BARRIER
+};
+
/* Describe state of dependencies used during sched_analyze phase. */
struct deps
{
/* Element N is set for each register that is conditionally set. */
regset_head reg_conditional_sets;
+
+ /* Shows the last value of reg_pending_barrier associated with the insn. */
+ enum reg_pending_barrier_mode last_reg_pending_barrier;
+
+ /* True when this context should be treated as a readonly by
+ the analysis. */
+ BOOL_BITFIELD readonly : 1;
};
+typedef struct deps *deps_t;
+
/* This structure holds some state of the current scheduling pass, and
contains some function pointers that abstract out some of the non-generic
functionality from functions such as schedule_block or schedule_insn.
There is one global variable, current_sched_info, which points to the
sched_info structure currently in use. */
-struct sched_info
+struct haifa_sched_info
{
/* Add all insns that are initially ready to the ready list. Called once
before scheduling a set of insns. */
necessary to identify this insn in an output. It's valid to use a
static buffer for this. The ALIGNED parameter should cause the string
to be formatted so that multiple output lines will line up nicely. */
- const char *(*print_insn) (rtx, int);
+ const char *(*print_insn) (const_rtx, int);
/* Return nonzero if an insn should be included in priority
calculations. */
int (*contributes_to_priority) (rtx, rtx);
- /* Called when computing dependencies for a JUMP_INSN. This function
- should store the set of registers that must be considered as set by
- the jump in the regset. */
- void (*compute_jump_reg_dependencies) (rtx, regset, regset, regset);
/* The boundaries of the set of insns to be scheduled. */
rtx prev_head, next_tail;
/* If nonzero, enables an additional sanity check in schedule_block. */
unsigned int queue_must_finish_empty:1;
- /* Nonzero if we should use cselib for better alias analysis. This
- must be 0 if the dependency information is used after sched_analyze
- has completed, e.g. if we're using it to initialize state for successor
- blocks in region scheduling. */
- unsigned int use_cselib:1;
/* Maximum priority that has been assigned to an insn. */
int sched_max_insns_priority;
last scheduled instruction. */
void (*begin_schedule_ready) (rtx, rtx);
- /* Called to notify frontend, that new basic block is being added.
- The first parameter - new basic block.
- The second parameter - block, after which new basic block is being added,
- or EXIT_BLOCK_PTR, if recovery block is being added,
- or NULL, if standalone block is being added. */
- void (*add_block) (basic_block, basic_block);
-
/* If the second parameter is not NULL, return nonnull value, if the
basic block should be advanced.
If the second parameter is NULL, return the next basic block in EBB.
The first parameter is the current basic block in EBB. */
basic_block (*advance_target_bb) (basic_block, rtx);
- /* Called after blocks were rearranged due to movement of jump instruction.
- The first parameter - index of basic block, in which jump currently is.
- The second parameter - index of basic block, in which jump used
- to be.
- The third parameter - index of basic block, that follows the second
- parameter. */
- void (*fix_recovery_cfg) (int, int, int);
-
/* ??? FIXME: should use straight bitfields inside sched_info instead of
this flag field. */
unsigned int flags;
/* Minimal cumulative weakness of speculative instruction's
dependencies, so that insn will be scheduled. */
- dw_t weakness_cutoff;
+ dw_t data_weakness_cutoff;
+
+ /* Minimal usefulness of speculative instruction to be considered for
+ scheduling. */
+ int control_weakness_cutoff;
/* Flags from the enum SPEC_SCHED_FLAGS. */
int flags;
};
typedef struct spec_info_def *spec_info_t;
-extern struct sched_info *current_sched_info;
+extern spec_info_t spec_info;
+
+extern struct haifa_sched_info *current_sched_info;
/* Indexed by INSN_UID, the collection of all data associated with
a single instruction. */
-struct haifa_insn_data
+struct _haifa_deps_insn_data
{
- /* We can't place 'struct _deps_list' into h_i_d instead of deps_list_t
- because when h_i_d extends, addresses of the deps_list->first
- change without updating deps_list->first->next->prev_nextp. */
+ /* The number of incoming edges in the forward dependency graph.
+ As scheduling proceeds, counts are decreased. An insn moves to
+ the ready queue when its counter reaches zero. */
+ int dep_count;
+
+ /* Nonzero if instruction has internal dependence
+ (e.g. add_dependence was invoked with (insn == elem)). */
+ unsigned int has_internal_dep;
+
+ /* NB: We can't place 'struct _deps_list' here instead of deps_list_t into
+ h_i_d because when h_i_d extends, addresses of the deps_list->first
+ change without updating deps_list->first->next->prev_nextp. Thus
+ BACK_DEPS and RESOLVED_BACK_DEPS are allocated on the heap and FORW_DEPS
+ list is allocated on the obstack. */
/* A list of hard backward dependencies. The insn is a consumer of all the
deps mentioned here. */
from 'forw_deps' to 'resolved_forw_deps' while scheduling to fasten the
search in 'forw_deps'. */
deps_list_t resolved_forw_deps;
-
+
+ /* Some insns (e.g. call) are not allowed to move across blocks. */
+ unsigned int cant_move : 1;
+};
+
+struct _haifa_insn_data
+{
+ /* We can't place 'struct _deps_list' into h_i_d instead of deps_list_t
+ because when h_i_d extends, addresses of the deps_list->first
+ change without updating deps_list->first->next->prev_nextp. */
+
/* Logical uid gives the original ordering of the insns. */
int luid;
register pressure. */
short reg_weight;
- /* Some insns (e.g. call) are not allowed to move across blocks. */
- unsigned int cant_move : 1;
-
/* Set if there's DEF-USE dependence between some speculatively
moved load insn and this one. */
unsigned int fed_by_spec_load : 1;
'< 0' if priority in invalid and should be recomputed. */
signed char priority_status;
- /* Nonzero if instruction has internal dependence
- (e.g. add_dependence was invoked with (insn == elem)). */
- unsigned int has_internal_dep : 1;
-
/* What speculations are necessary to apply to schedule the instruction. */
ds_t todo_spec;
+
/* What speculations were already applied. */
ds_t done_spec;
+
/* What speculations are checked by this instruction. */
ds_t check_spec;
rtx orig_pat;
};
-extern struct haifa_insn_data *h_i_d;
+typedef struct _haifa_insn_data haifa_insn_data_def;
+typedef haifa_insn_data_def *haifa_insn_data_t;
+
+DEF_VEC_O (haifa_insn_data_def);
+DEF_VEC_ALLOC_O (haifa_insn_data_def, heap);
+
+extern VEC(haifa_insn_data_def, heap) *h_i_d;
+
+#define HID(INSN) (VEC_index (haifa_insn_data_def, h_i_d, INSN_UID (INSN)))
/* Accessor macros for h_i_d. There are more in haifa-sched.c and
sched-rgn.c. */
-
-#define INSN_HARD_BACK_DEPS(INSN) (h_i_d[INSN_UID (INSN)].hard_back_deps)
-#define INSN_SPEC_BACK_DEPS(INSN) (h_i_d[INSN_UID (INSN)].spec_back_deps)
-#define INSN_FORW_DEPS(INSN) (h_i_d[INSN_UID (INSN)].forw_deps)
-#define INSN_RESOLVED_BACK_DEPS(INSN) \
- (h_i_d[INSN_UID (INSN)].resolved_back_deps)
-#define INSN_RESOLVED_FORW_DEPS(INSN) \
- (h_i_d[INSN_UID (INSN)].resolved_forw_deps)
-#define INSN_LUID(INSN) (h_i_d[INSN_UID (INSN)].luid)
-#define CANT_MOVE(insn) (h_i_d[INSN_UID (insn)].cant_move)
-#define INSN_PRIORITY(INSN) (h_i_d[INSN_UID (INSN)].priority)
-#define INSN_PRIORITY_STATUS(INSN) (h_i_d[INSN_UID (INSN)].priority_status)
+#define INSN_PRIORITY(INSN) (HID (INSN)->priority)
+#define INSN_REG_WEIGHT(INSN) (HID (INSN)->reg_weight)
+#define INSN_PRIORITY_STATUS(INSN) (HID (INSN)->priority_status)
+
+typedef struct _haifa_deps_insn_data haifa_deps_insn_data_def;
+typedef haifa_deps_insn_data_def *haifa_deps_insn_data_t;
+
+DEF_VEC_O (haifa_deps_insn_data_def);
+DEF_VEC_ALLOC_O (haifa_deps_insn_data_def, heap);
+
+extern VEC(haifa_deps_insn_data_def, heap) *h_d_i_d;
+
+#define HDID(INSN) (VEC_index (haifa_deps_insn_data_def, h_d_i_d, \
+ INSN_LUID (INSN)))
+#define INSN_DEP_COUNT(INSN) (HDID (INSN)->dep_count)
+#define HAS_INTERNAL_DEP(INSN) (HDID (INSN)->has_internal_dep)
+#define INSN_FORW_DEPS(INSN) (HDID (INSN)->forw_deps)
+#define INSN_RESOLVED_BACK_DEPS(INSN) (HDID (INSN)->resolved_back_deps)
+#define INSN_RESOLVED_FORW_DEPS(INSN) (HDID (INSN)->resolved_forw_deps)
+#define INSN_HARD_BACK_DEPS(INSN) (HDID (INSN)->hard_back_deps)
+#define INSN_SPEC_BACK_DEPS(INSN) (HDID (INSN)->spec_back_deps)
+#define CANT_MOVE(INSN) (HDID (INSN)->cant_move)
+#define CANT_MOVE_BY_LUID(LUID) (VEC_index (haifa_deps_insn_data_def, h_d_i_d, \
+ LUID)->cant_move)
+
+
+#define INSN_PRIORITY(INSN) (HID (INSN)->priority)
+#define INSN_PRIORITY_STATUS(INSN) (HID (INSN)->priority_status)
#define INSN_PRIORITY_KNOWN(INSN) (INSN_PRIORITY_STATUS (INSN) > 0)
-#define INSN_REG_WEIGHT(INSN) (h_i_d[INSN_UID (INSN)].reg_weight)
-#define HAS_INTERNAL_DEP(INSN) (h_i_d[INSN_UID (INSN)].has_internal_dep)
-#define TODO_SPEC(INSN) (h_i_d[INSN_UID (INSN)].todo_spec)
-#define DONE_SPEC(INSN) (h_i_d[INSN_UID (INSN)].done_spec)
-#define CHECK_SPEC(INSN) (h_i_d[INSN_UID (INSN)].check_spec)
-#define RECOVERY_BLOCK(INSN) (h_i_d[INSN_UID (INSN)].recovery_block)
-#define ORIG_PAT(INSN) (h_i_d[INSN_UID (INSN)].orig_pat)
+#define TODO_SPEC(INSN) (HID (INSN)->todo_spec)
+#define DONE_SPEC(INSN) (HID (INSN)->done_spec)
+#define CHECK_SPEC(INSN) (HID (INSN)->check_spec)
+#define RECOVERY_BLOCK(INSN) (HID (INSN)->recovery_block)
+#define ORIG_PAT(INSN) (HID (INSN)->orig_pat)
/* INSN is either a simple or a branchy speculation check. */
-#define IS_SPECULATION_CHECK_P(INSN) (RECOVERY_BLOCK (INSN) != NULL)
+#define IS_SPECULATION_CHECK_P(INSN) \
+ (sel_sched_p () ? sel_insn_is_speculation_check (INSN) : RECOVERY_BLOCK (INSN) != NULL)
/* INSN is a speculation check that will simply reexecute the speculatively
scheduled instruction if the speculation fails. */
DO_SPECULATION = USE_DEPS_LIST << 1,
SCHED_RGN = DO_SPECULATION << 1,
SCHED_EBB = SCHED_RGN << 1,
- /* Scheduler can possible create new basic blocks. Used for assertions. */
- NEW_BBS = SCHED_EBB << 1
+ /* Scheduler can possibly create new basic blocks. Used for assertions. */
+ NEW_BBS = SCHED_EBB << 1,
+ SEL_SCHED = NEW_BBS << 1
};
enum SPEC_SCHED_FLAGS {
COUNT_SPEC_IN_CRITICAL_PATH = 1,
PREFER_NON_DATA_SPEC = COUNT_SPEC_IN_CRITICAL_PATH << 1,
- PREFER_NON_CONTROL_SPEC = PREFER_NON_DATA_SPEC << 1
+ PREFER_NON_CONTROL_SPEC = PREFER_NON_DATA_SPEC << 1,
+ SEL_SCHED_SPEC_DONT_CHECK_CONTROL = PREFER_NON_CONTROL_SPEC << 1
};
#define NOTE_NOT_BB_P(NOTE) (NOTE_P (NOTE) && (NOTE_KIND (NOTE) \
#define HAIFA_INLINE __inline
#endif
+struct sched_deps_info_def
+{
+ /* Called when computing dependencies for a JUMP_INSN. This function
+ should store the set of registers that must be considered as set by
+ the jump in the regset. */
+ void (*compute_jump_reg_dependencies) (rtx, regset, regset, regset);
+
+ /* Start analyzing insn. */
+ void (*start_insn) (rtx);
+
+ /* Finish analyzing insn. */
+ void (*finish_insn) (void);
+
+ /* Start analyzing insn LHS (Left Hand Side). */
+ void (*start_lhs) (rtx);
+
+ /* Finish analyzing insn LHS. */
+ void (*finish_lhs) (void);
+
+ /* Start analyzing insn RHS (Right Hand Side). */
+ void (*start_rhs) (rtx);
+
+ /* Finish analyzing insn RHS. */
+ void (*finish_rhs) (void);
+
+ /* Note set of the register. */
+ void (*note_reg_set) (int);
+
+ /* Note clobber of the register. */
+ void (*note_reg_clobber) (int);
+
+ /* Note use of the register. */
+ void (*note_reg_use) (int);
+
+ /* Note memory dependence of type DS between MEM1 and MEM2 (which is
+ in the INSN2). */
+ void (*note_mem_dep) (rtx mem1, rtx mem2, rtx insn2, ds_t ds);
+
+ /* Note a dependence of type DS from the INSN. */
+ void (*note_dep) (rtx insn, ds_t ds);
+
+ /* Nonzero if we should use cselib for better alias analysis. This
+ must be 0 if the dependency information is used after sched_analyze
+ has completed, e.g. if we're using it to initialize state for successor
+ blocks in region scheduling. */
+ unsigned int use_cselib : 1;
+
+ /* If set, generate links between instruction as DEPS_LIST.
+ Otherwise, generate usual INSN_LIST links. */
+ unsigned int use_deps_list : 1;
+
+ /* Generate data and control speculative dependencies.
+ Requires USE_DEPS_LIST set. */
+ unsigned int generate_spec_deps : 1;
+};
+
+extern struct sched_deps_info_def *sched_deps_info;
+
+
/* Functions in sched-deps.c. */
extern bool sched_insns_conditions_mutex_p (const_rtx, const_rtx);
extern bool sched_insn_is_legitimate_for_speculation_p (const_rtx, ds_t);
extern void add_dependence (rtx, rtx, enum reg_note);
extern void sched_analyze (struct deps *, rtx, rtx);
-extern bool deps_pools_are_empty_p (void);
-extern void sched_free_deps (rtx, rtx, bool);
extern void init_deps (struct deps *);
extern void free_deps (struct deps *);
extern void init_deps_global (void);
extern void finish_deps_global (void);
-extern void init_dependency_caches (int);
-extern void free_dependency_caches (void);
-extern void extend_dependency_caches (int, bool);
+extern void deps_analyze_insn (struct deps *, rtx);
+extern void remove_from_deps (struct deps *, rtx);
+
+extern dw_t get_dep_weak_1 (ds_t, ds_t);
extern dw_t get_dep_weak (ds_t, ds_t);
extern ds_t set_dep_weak (ds_t, ds_t, dw_t);
+extern dw_t estimate_dep_weak (rtx, rtx);
extern ds_t ds_merge (ds_t, ds_t);
+extern ds_t ds_full_merge (ds_t, ds_t, rtx, rtx);
+extern ds_t ds_max_merge (ds_t, ds_t);
+extern dw_t ds_weak (ds_t);
+extern ds_t ds_get_speculation_types (ds_t);
+extern ds_t ds_get_max_dep_weak (ds_t);
+
+extern void sched_deps_init (bool);
+extern void sched_deps_finish (void);
+
+extern void haifa_note_reg_set (int);
+extern void haifa_note_reg_clobber (int);
+extern void haifa_note_reg_use (int);
+
+extern void maybe_extend_reg_info_p (void);
+
+extern void deps_start_bb (struct deps *, rtx);
+extern enum reg_note ds_to_dt (ds_t);
+
+extern bool deps_pools_are_empty_p (void);
+extern void sched_free_deps (rtx, rtx, bool);
+extern void extend_dependency_caches (int, bool);
+
extern void debug_ds (ds_t);
/* Functions in haifa-sched.c. */
extern void get_ebb_head_tail (basic_block, basic_block, rtx *, rtx *);
extern int no_real_insns_p (const_rtx, const_rtx);
-extern void rm_other_notes (rtx, rtx);
-
extern int insn_cost (rtx);
+extern int dep_cost_1 (dep_t, dw_t);
extern int dep_cost (dep_t);
extern int set_priorities (rtx, rtx);
-extern void schedule_block (basic_block *, int);
-extern void sched_init (void);
-extern void sched_finish (void);
+extern void schedule_block (basic_block *);
+
+extern int cycle_issued_insns;
+extern int issue_rate;
+extern int dfa_lookahead;
+
+extern void ready_sort (struct ready_list *);
+extern rtx ready_element (struct ready_list *, int);
+extern rtx *ready_lastpos (struct ready_list *);
extern int try_ready (rtx);
-extern void * xrecalloc (void *, size_t, size_t, size_t);
+extern void sched_extend_ready_list (int);
+extern void sched_finish_ready_list (void);
+extern void sched_change_pattern (rtx, rtx);
+extern int sched_speculate_insn (rtx, ds_t, rtx *);
extern void unlink_bb_notes (basic_block, basic_block);
extern void add_block (basic_block, basic_block);
extern rtx bb_note (basic_block);
+extern void concat_note_lists (rtx, rtx *);
+\f
-/* Functions in sched-rgn.c. */
+/* Types and functions in sched-rgn.c. */
+/* A region is the main entity for interblock scheduling: insns
+ are allowed to move between blocks in the same region, along
+ control flow graph edges, in the 'up' direction. */
+typedef struct
+{
+ /* Number of extended basic blocks in region. */
+ int rgn_nr_blocks;
+ /* cblocks in the region (actually index in rgn_bb_table). */
+ int rgn_blocks;
+ /* Dependencies for this region are already computed. Basically, indicates,
+ that this is a recovery block. */
+ unsigned int dont_calc_deps : 1;
+ /* This region has at least one non-trivial ebb. */
+ unsigned int has_real_ebb : 1;
+}
+region;
+
+extern int nr_regions;
+extern region *rgn_table;
+extern int *rgn_bb_table;
+extern int *block_to_bb;
+extern int *containing_rgn;
+
+#define RGN_NR_BLOCKS(rgn) (rgn_table[rgn].rgn_nr_blocks)
+#define RGN_BLOCKS(rgn) (rgn_table[rgn].rgn_blocks)
+#define RGN_DONT_CALC_DEPS(rgn) (rgn_table[rgn].dont_calc_deps)
+#define RGN_HAS_REAL_EBB(rgn) (rgn_table[rgn].has_real_ebb)
+#define BLOCK_TO_BB(block) (block_to_bb[block])
+#define CONTAINING_RGN(block) (containing_rgn[block])
+
+/* The mapping from ebb to block. */
+extern int *ebb_head;
+#define BB_TO_BLOCK(ebb) (rgn_bb_table[ebb_head[ebb]])
+#define EBB_FIRST_BB(ebb) BASIC_BLOCK (BB_TO_BLOCK (ebb))
+#define EBB_LAST_BB(ebb) BASIC_BLOCK (rgn_bb_table[ebb_head[ebb + 1] - 1])
+#define INSN_BB(INSN) (BLOCK_TO_BB (BLOCK_NUM (INSN)))
+
+extern int current_nr_blocks;
+extern int current_blocks;
+extern int target_bb;
+
+extern bool sched_is_disabled_for_current_region_p (void);
+extern void sched_rgn_init (bool);
+extern void sched_rgn_finish (void);
+extern void rgn_setup_region (int);
+extern void sched_rgn_compute_dependencies (int);
+extern void sched_rgn_local_init (int);
+extern void sched_rgn_local_finish (void);
+extern void sched_rgn_local_free (void);
+extern void extend_regions (void);
+extern void rgn_make_new_region_out_of_new_block (basic_block);
+
+extern void compute_priorities (void);
+extern void increase_insn_priority (rtx, int);
+extern void debug_rgn_dependencies (int);
extern void debug_dependencies (rtx, rtx);
+extern void free_rgn_deps (void);
+extern int contributes_to_priority (rtx, rtx);
+extern void extend_rgns (int *, int *, sbitmap, int *);
+extern void deps_join (struct deps *, struct deps *);
+
+extern void rgn_setup_common_sched_info (void);
+extern void rgn_setup_sched_infos (void);
+
+extern void debug_regions (void);
+extern void debug_region (int);
+extern void dump_region_dot (FILE *, int);
+extern void dump_region_dot_file (const char *, int);
+
+extern void haifa_sched_init (void);
+extern void haifa_sched_finish (void);
/* sched-deps.c interface to walk, add, search, update, resolve, delete
and debug instruction dependencies. */
#include "cfglayout.h"
#include "params.h"
#include "sched-int.h"
+#include "sel-sched.h"
#include "target.h"
#include "timevar.h"
#include "tree-pass.h"
#include "dbgcnt.h"
#ifdef INSN_SCHEDULING
+
/* Some accessor macros for h_i_d members only used within this file. */
-#define INSN_REF_COUNT(INSN) (h_i_d[INSN_UID (INSN)].ref_count)
-#define FED_BY_SPEC_LOAD(insn) (h_i_d[INSN_UID (insn)].fed_by_spec_load)
-#define IS_LOAD_INSN(insn) (h_i_d[INSN_UID (insn)].is_load_insn)
+#define FED_BY_SPEC_LOAD(INSN) (HID (INSN)->fed_by_spec_load)
+#define IS_LOAD_INSN(INSN) (HID (insn)->is_load_insn)
/* nr_inter/spec counts interblock/speculative motion for the function. */
static int nr_inter, nr_spec;
static int is_cfg_nonregular (void);
-static bool sched_is_disabled_for_current_region_p (void);
-
-/* A region is the main entity for interblock scheduling: insns
- are allowed to move between blocks in the same region, along
- control flow graph edges, in the 'up' direction. */
-typedef struct
-{
- /* Number of extended basic blocks in region. */
- int rgn_nr_blocks;
- /* cblocks in the region (actually index in rgn_bb_table). */
- int rgn_blocks;
- /* Dependencies for this region are already computed. Basically, indicates,
- that this is a recovery block. */
- unsigned int dont_calc_deps : 1;
- /* This region has at least one non-trivial ebb. */
- unsigned int has_real_ebb : 1;
-}
-region;
/* Number of regions in the procedure. */
-static int nr_regions;
+int nr_regions = 0;
/* Table of region descriptions. */
-static region *rgn_table;
+region *rgn_table = NULL;
/* Array of lists of regions' blocks. */
-static int *rgn_bb_table;
+int *rgn_bb_table = NULL;
/* Topological order of blocks in the region (if b2 is reachable from
b1, block_to_bb[b2] > block_to_bb[b1]). Note: A basic block is
always referred to by either block or b, while its topological
order name (in the region) is referred to by bb. */
-static int *block_to_bb;
+int *block_to_bb = NULL;
/* The number of the region containing a block. */
-static int *containing_rgn;
+int *containing_rgn = NULL;
+
+/* ebb_head [i] - is index in rgn_bb_table of the head basic block of i'th ebb.
+ Currently we can get a ebb only through splitting of currently
+ scheduling block, therefore, we don't need ebb_head array for every region,
+ hence, its sufficient to hold it for current one only. */
+int *ebb_head = NULL;
/* The minimum probability of reaching a source block so that it will be
considered for speculative scheduling. */
static int min_spec_prob;
-#define RGN_NR_BLOCKS(rgn) (rgn_table[rgn].rgn_nr_blocks)
-#define RGN_BLOCKS(rgn) (rgn_table[rgn].rgn_blocks)
-#define RGN_DONT_CALC_DEPS(rgn) (rgn_table[rgn].dont_calc_deps)
-#define RGN_HAS_REAL_EBB(rgn) (rgn_table[rgn].has_real_ebb)
-#define BLOCK_TO_BB(block) (block_to_bb[block])
-#define CONTAINING_RGN(block) (containing_rgn[block])
-
-void debug_regions (void);
-static void find_single_block_region (void);
+static void find_single_block_region (bool);
static void find_rgns (void);
-static void extend_rgns (int *, int *, sbitmap, int *);
static bool too_large (int, int *, int *);
-extern void debug_live (int, int);
-
/* Blocks of the current region being scheduled. */
-static int current_nr_blocks;
-static int current_blocks;
+int current_nr_blocks;
+int current_blocks;
-static int rgn_n_insns;
+/* A speculative motion requires checking live information on the path
+ from 'source' to 'target'. The split blocks are those to be checked.
+ After a speculative motion, live information should be modified in
+ the 'update' blocks.
-/* The mapping from ebb to block. */
-/* ebb_head [i] - is index in rgn_bb_table, while
- EBB_HEAD (i) - is basic block index.
- BASIC_BLOCK (EBB_HEAD (i)) - head of ebb. */
-#define BB_TO_BLOCK(ebb) (rgn_bb_table[ebb_head[ebb]])
-#define EBB_FIRST_BB(ebb) BASIC_BLOCK (BB_TO_BLOCK (ebb))
-#define EBB_LAST_BB(ebb) BASIC_BLOCK (rgn_bb_table[ebb_head[ebb + 1] - 1])
+ Lists of split and update blocks for each candidate of the current
+ target are in array bblst_table. */
+static basic_block *bblst_table;
+static int bblst_size, bblst_last;
/* Target info declarations.
candidate;
static candidate *candidate_table;
-
-/* A speculative motion requires checking live information on the path
- from 'source' to 'target'. The split blocks are those to be checked.
- After a speculative motion, live information should be modified in
- the 'update' blocks.
-
- Lists of split and update blocks for each candidate of the current
- target are in array bblst_table. */
-static basic_block *bblst_table;
-static int bblst_size, bblst_last;
-
-#define IS_VALID(src) ( candidate_table[src].is_valid )
-#define IS_SPECULATIVE(src) ( candidate_table[src].is_speculative )
+#define IS_VALID(src) (candidate_table[src].is_valid)
+#define IS_SPECULATIVE(src) (candidate_table[src].is_speculative)
+#define IS_SPECULATIVE_INSN(INSN) \
+ (IS_SPECULATIVE (BLOCK_TO_BB (BLOCK_NUM (INSN))))
#define SRC_PROB(src) ( candidate_table[src].src_prob )
/* The bb being currently scheduled. */
-static int target_bb;
+int target_bb;
/* List of edges. */
typedef struct
static void extract_edgelst (sbitmap, edgelst *);
-
/* Target info functions. */
static void split_edges (int, int, edgelst *);
static void compute_trg_info (int);
/* For every bb, a set of its ancestor edges. */
static edgeset *ancestor_edges;
-/* Array of EBBs sizes. Currently we can get a ebb only through
- splitting of currently scheduling block, therefore, we don't need
- ebb_head array for every region, its sufficient to hold it only
- for current one. */
-static int *ebb_head;
-
-static void compute_dom_prob_ps (int);
-
#define INSN_PROBABILITY(INSN) (SRC_PROB (BLOCK_TO_BB (BLOCK_NUM (INSN))))
-#define IS_SPECULATIVE_INSN(INSN) (IS_SPECULATIVE (BLOCK_TO_BB (BLOCK_NUM (INSN))))
-#define INSN_BB(INSN) (BLOCK_TO_BB (BLOCK_NUM (INSN)))
/* Speculative scheduling functions. */
static int check_live_1 (int, rtx);
static void update_live_1 (int, rtx);
-static int check_live (rtx, int);
-static void update_live (rtx, int);
-static void set_spec_fed (rtx);
static int is_pfree (rtx, int, int);
static int find_conditional_protection (rtx, int);
static int is_conditionally_protected (rtx, int, int);
static void add_branch_dependences (rtx, rtx);
static void compute_block_dependences (int);
-static void init_regions (void);
static void schedule_region (int);
static rtx concat_INSN_LIST (rtx, rtx);
static void concat_insn_mem_list (rtx, rtx, rtx *, rtx *);
}
}
+/* Print the region's basic blocks. */
+
+void
+debug_region (int rgn)
+{
+ int bb;
+
+ fprintf (stderr, "\n;; ------------ REGION %d ----------\n\n", rgn);
+ fprintf (stderr, ";;\trgn %d nr_blocks %d:\n", rgn,
+ rgn_table[rgn].rgn_nr_blocks);
+ fprintf (stderr, ";;\tbb/block: ");
+
+ /* We don't have ebb_head initialized yet, so we can't use
+ BB_TO_BLOCK (). */
+ current_blocks = RGN_BLOCKS (rgn);
+
+ for (bb = 0; bb < rgn_table[rgn].rgn_nr_blocks; bb++)
+ fprintf (stderr, " %d/%d ", bb, rgn_bb_table[current_blocks + bb]);
+
+ fprintf (stderr, "\n\n");
+
+ for (bb = 0; bb < rgn_table[rgn].rgn_nr_blocks; bb++)
+ {
+ debug_bb_n_slim (rgn_bb_table[current_blocks + bb]);
+ fprintf (stderr, "\n");
+ }
+
+ fprintf (stderr, "\n");
+
+}
+
+/* True when a bb with index BB_INDEX contained in region RGN. */
+static bool
+bb_in_region_p (int bb_index, int rgn)
+{
+ int i;
+
+ for (i = 0; i < rgn_table[rgn].rgn_nr_blocks; i++)
+ if (rgn_bb_table[current_blocks + i] == bb_index)
+ return true;
+
+ return false;
+}
+
+/* Dump region RGN to file F using dot syntax. */
+void
+dump_region_dot (FILE *f, int rgn)
+{
+ int i;
+
+ fprintf (f, "digraph Region_%d {\n", rgn);
+
+ /* We don't have ebb_head initialized yet, so we can't use
+ BB_TO_BLOCK (). */
+ current_blocks = RGN_BLOCKS (rgn);
+
+ for (i = 0; i < rgn_table[rgn].rgn_nr_blocks; i++)
+ {
+ edge e;
+ edge_iterator ei;
+ int src_bb_num = rgn_bb_table[current_blocks + i];
+ struct basic_block_def *bb = BASIC_BLOCK (src_bb_num);
+
+ FOR_EACH_EDGE (e, ei, bb->succs)
+ if (bb_in_region_p (e->dest->index, rgn))
+ fprintf (f, "\t%d -> %d\n", src_bb_num, e->dest->index);
+ }
+ fprintf (f, "}\n");
+}
+
+/* The same, but first open a file specified by FNAME. */
+void
+dump_region_dot_file (const char *fname, int rgn)
+{
+ FILE *f = fopen (fname, "wt");
+ dump_region_dot (f, rgn);
+ fclose (f);
+}
+
/* Build a single block region for each basic block in the function.
This allows for using the same code for interblock and basic block
scheduling. */
static void
-find_single_block_region (void)
+find_single_block_region (bool ebbs_p)
{
- basic_block bb;
+ basic_block bb, ebb_start;
+ int i = 0;
nr_regions = 0;
- FOR_EACH_BB (bb)
- {
- rgn_bb_table[nr_regions] = bb->index;
- RGN_NR_BLOCKS (nr_regions) = 1;
- RGN_BLOCKS (nr_regions) = nr_regions;
- RGN_DONT_CALC_DEPS (nr_regions) = 0;
- RGN_HAS_REAL_EBB (nr_regions) = 0;
- CONTAINING_RGN (bb->index) = nr_regions;
- BLOCK_TO_BB (bb->index) = 0;
- nr_regions++;
- }
+ if (ebbs_p) {
+ int probability_cutoff;
+ if (profile_info && flag_branch_probabilities)
+ probability_cutoff = PARAM_VALUE (TRACER_MIN_BRANCH_PROBABILITY_FEEDBACK);
+ else
+ probability_cutoff = PARAM_VALUE (TRACER_MIN_BRANCH_PROBABILITY);
+ probability_cutoff = REG_BR_PROB_BASE / 100 * probability_cutoff;
+
+ FOR_EACH_BB (ebb_start)
+ {
+ RGN_NR_BLOCKS (nr_regions) = 0;
+ RGN_BLOCKS (nr_regions) = i;
+ RGN_DONT_CALC_DEPS (nr_regions) = 0;
+ RGN_HAS_REAL_EBB (nr_regions) = 0;
+
+ for (bb = ebb_start; ; bb = bb->next_bb)
+ {
+ edge e;
+ edge_iterator ei;
+
+ rgn_bb_table[i] = bb->index;
+ RGN_NR_BLOCKS (nr_regions)++;
+ CONTAINING_RGN (bb->index) = nr_regions;
+ BLOCK_TO_BB (bb->index) = i - RGN_BLOCKS (nr_regions);
+ i++;
+
+ if (bb->next_bb == EXIT_BLOCK_PTR
+ || LABEL_P (BB_HEAD (bb->next_bb)))
+ break;
+
+ FOR_EACH_EDGE (e, ei, bb->succs)
+ if ((e->flags & EDGE_FALLTHRU) != 0)
+ break;
+ if (! e)
+ break;
+ if (e->probability <= probability_cutoff)
+ break;
+ }
+
+ ebb_start = bb;
+ nr_regions++;
+ }
+ }
+ else
+ FOR_EACH_BB (bb)
+ {
+ rgn_bb_table[nr_regions] = bb->index;
+ RGN_NR_BLOCKS (nr_regions) = 1;
+ RGN_BLOCKS (nr_regions) = nr_regions;
+ RGN_DONT_CALC_DEPS (nr_regions) = 0;
+ RGN_HAS_REAL_EBB (nr_regions) = 0;
+
+ CONTAINING_RGN (bb->index) = nr_regions;
+ BLOCK_TO_BB (bb->index) = 0;
+ nr_regions++;
+ }
+}
+
+/* Estimate number of the insns in the BB. */
+static int
+rgn_estimate_number_of_insns (basic_block bb)
+{
+ return INSN_LUID (BB_END (bb)) - INSN_LUID (BB_HEAD (bb));
}
/* Update number of blocks and the estimate for number of insns
too_large (int block, int *num_bbs, int *num_insns)
{
(*num_bbs)++;
- (*num_insns) += (INSN_LUID (BB_END (BASIC_BLOCK (block)))
- - INSN_LUID (BB_HEAD (BASIC_BLOCK (block))));
+ (*num_insns) += (common_sched_info->estimate_number_of_insns
+ (BASIC_BLOCK (block)));
return ((*num_bbs > PARAM_VALUE (PARAM_MAX_SCHED_REGION_BLOCKS))
|| (*num_insns > PARAM_VALUE (PARAM_MAX_SCHED_REGION_INSNS)));
of edge tables. That would simplify it somewhat. */
static void
-find_rgns (void)
+haifa_find_rgns (void)
{
int *max_hdr, *dfs_nr, *degree;
char no_loops = 1;
/* Estimate # insns, and count # blocks in the region. */
num_bbs = 1;
- num_insns = (INSN_LUID (BB_END (bb))
- - INSN_LUID (BB_HEAD (bb)));
+ num_insns = common_sched_info->estimate_number_of_insns (bb);
/* Find all loop latches (blocks with back edges to the loop
header) or all the leaf blocks in the cfg has no loops.
sbitmap_free (in_stack);
}
+
+/* Wrapper function.
+ If FLAG_SEL_SCHED_PIPELINING is set, then use custom function to form
+ regions. Otherwise just call find_rgns_haifa. */
+static void
+find_rgns (void)
+{
+ if (sel_sched_p () && flag_sel_sched_pipelining)
+ sel_find_rgns ();
+ else
+ haifa_find_rgns ();
+}
+
static int gather_region_statistics (int **);
static void print_region_statistics (int *, int, int *, int);
LOOP_HDR - mapping from block to the containing loop
(two blocks can reside within one region if they have
the same loop header). */
-static void
+void
extend_rgns (int *degree, int *idxp, sbitmap header, int *loop_hdr)
{
int *order, i, rescan = 0, idx = *idxp, iter = 0, max_iter, *max_hdr;
CFG should be traversed until no further changes are made. On each
iteration the set of the region heads is extended (the set of those
blocks that have max_hdr[bbi] == bbi). This set is upper bounded by the
- set of all basic blocks, thus the algorithm is guaranteed to terminate. */
+ set of all basic blocks, thus the algorithm is guaranteed to
+ terminate. */
while (rescan && iter < max_iter)
{
edge_iterator ei;
edge e;
+ candidate_table = XNEWVEC (candidate, current_nr_blocks);
+
+ bblst_last = 0;
+ /* bblst_table holds split blocks and update blocks for each block after
+ the current one in the region. split blocks and update blocks are
+ the TO blocks of region edges, so there can be at most rgn_nr_edges
+ of them. */
+ bblst_size = (current_nr_blocks - target_bb) * rgn_nr_edges;
+ bblst_table = XNEWVEC (basic_block, bblst_size);
+
+ edgelst_last = 0;
+ edgelst_table = XNEWVEC (edge, rgn_nr_edges);
+
/* Define some of the fields for the target bb as well. */
sp = candidate_table + trg;
sp->is_valid = 1;
sbitmap_free (visited);
}
+/* Free the computed target info. */
+static void
+free_trg_info (void)
+{
+ free (candidate_table);
+ free (bblst_table);
+ free (edgelst_table);
+}
+
/* Print candidates info, for debugging purposes. Callable from debugger. */
void
static void begin_schedule_ready (rtx, rtx);
static ds_t new_ready (rtx, ds_t);
static int schedule_more_p (void);
-static const char *rgn_print_insn (rtx, int);
+static const char *rgn_print_insn (const_rtx, int);
static int rgn_rank (rtx, rtx);
-static int contributes_to_priority (rtx, rtx);
static void compute_jump_reg_dependencies (rtx, regset, regset, regset);
/* Functions for speculative scheduling. */
-static void add_remove_insn (rtx, int);
-static void extend_regions (void);
-static void add_block1 (basic_block, basic_block);
-static void fix_recovery_cfg (int, int, int);
+static void rgn_add_remove_insn (rtx, int);
+static void rgn_add_block (basic_block, basic_block);
+static void rgn_fix_recovery_cfg (int, int, int);
static basic_block advance_target_bb (basic_block, rtx);
-static void debug_rgn_dependencies (int);
-
/* Return nonzero if there are more insns that should be scheduled. */
static int
/* Prepare current target block info. */
if (current_nr_blocks > 1)
- {
- candidate_table = XNEWVEC (candidate, current_nr_blocks);
-
- bblst_last = 0;
- /* bblst_table holds split blocks and update blocks for each block after
- the current one in the region. split blocks and update blocks are
- the TO blocks of region edges, so there can be at most rgn_nr_edges
- of them. */
- bblst_size = (current_nr_blocks - target_bb) * rgn_nr_edges;
- bblst_table = XNEWVEC (basic_block, bblst_size);
-
- edgelst_last = 0;
- edgelst_table = XNEWVEC (edge, rgn_nr_edges);
-
- compute_trg_info (target_bb);
- }
+ compute_trg_info (target_bb);
/* Initialize ready list with all 'ready' insns in target block.
Count number of insns in the target block being scheduled. */
if (not_ex_free
/* We are here because is_exception_free () == false.
But we possibly can handle that with control speculation. */
- && (current_sched_info->flags & DO_SPECULATION)
- && (spec_info->mask & BEGIN_CONTROL))
+ && sched_deps_info->generate_spec_deps
+ && spec_info->mask & BEGIN_CONTROL)
{
ds_t new_ds;
to be formatted so that multiple output lines will line up nicely. */
static const char *
-rgn_print_insn (rtx insn, int aligned)
+rgn_print_insn (const_rtx insn, int aligned)
{
static char tmp[80];
return nonzero if we should include this dependence in priority
calculations. */
-static int
+int
contributes_to_priority (rtx next, rtx insn)
{
/* NEXT and INSN reside in one ebb. */
add_branch_dependences. */
}
+/* This variable holds common_sched_info hooks and data relevant to
+ the interblock scheduler. */
+static struct common_sched_info_def rgn_common_sched_info;
+
+
+/* This holds data for the dependence analysis relevant to
+ the interblock scheduler. */
+static struct sched_deps_info_def rgn_sched_deps_info;
+
+/* This holds constant data used for initializing the above structure
+ for the Haifa scheduler. */
+static const struct sched_deps_info_def rgn_const_sched_deps_info =
+ {
+ compute_jump_reg_dependencies,
+ NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,
+ 0, 0, 0
+ };
+
+/* Same as above, but for the selective scheduler. */
+static const struct sched_deps_info_def rgn_const_sel_sched_deps_info =
+ {
+ compute_jump_reg_dependencies,
+ NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,
+ 0, 0, 0
+ };
+
/* Used in schedule_insns to initialize current_sched_info for scheduling
regions (or single basic blocks). */
-static struct sched_info region_sched_info =
+static const struct haifa_sched_info rgn_const_sched_info =
{
init_ready_list,
can_schedule_ready_p,
rgn_rank,
rgn_print_insn,
contributes_to_priority,
- compute_jump_reg_dependencies,
NULL, NULL,
NULL, NULL,
- 0, 0, 0,
+ 0, 0,
- add_remove_insn,
+ rgn_add_remove_insn,
begin_schedule_ready,
- add_block1,
advance_target_bb,
- fix_recovery_cfg,
SCHED_RGN
};
+/* This variable holds the data and hooks needed to the Haifa scheduler backend
+ for the interblock scheduler frontend. */
+static struct haifa_sched_info rgn_sched_info;
+
+/* Returns maximum priority that an insn was assigned to. */
+
+int
+get_rgn_sched_max_insns_priority (void)
+{
+ return rgn_sched_info.sched_max_insns_priority;
+}
+
/* Determine if PAT sets a CLASS_LIKELY_SPILLED_P register. */
static bool
*ret = true;
}
+/* An array used to hold the number of dependencies in which insn
+ participates. Used in add_branch_dependences. */
+static int *ref_counts;
+
/* Add dependences so that branches are scheduled to run last in their
block. */
-
static void
add_branch_dependences (rtx head, rtx tail)
{
are not moved before reload because we can wind up with register
allocation failures. */
+#define INSN_REF_COUNT(INSN) (ref_counts[INSN_UID (INSN)])
+
insn = tail;
last = 0;
while (CALL_P (insn)
*old_mems_p = new_mems;
}
+/* Join PRED_DEPS to the SUCC_DEPS. */
+void
+deps_join (struct deps *succ_deps, struct deps *pred_deps)
+{
+ unsigned reg;
+ reg_set_iterator rsi;
+
+ /* The reg_last lists are inherited by successor. */
+ EXECUTE_IF_SET_IN_REG_SET (&pred_deps->reg_last_in_use, 0, reg, rsi)
+ {
+ struct deps_reg *pred_rl = &pred_deps->reg_last[reg];
+ struct deps_reg *succ_rl = &succ_deps->reg_last[reg];
+
+ succ_rl->uses = concat_INSN_LIST (pred_rl->uses, succ_rl->uses);
+ succ_rl->sets = concat_INSN_LIST (pred_rl->sets, succ_rl->sets);
+ succ_rl->clobbers = concat_INSN_LIST (pred_rl->clobbers,
+ succ_rl->clobbers);
+ succ_rl->uses_length += pred_rl->uses_length;
+ succ_rl->clobbers_length += pred_rl->clobbers_length;
+ }
+ IOR_REG_SET (&succ_deps->reg_last_in_use, &pred_deps->reg_last_in_use);
+
+ /* Mem read/write lists are inherited by successor. */
+ concat_insn_mem_list (pred_deps->pending_read_insns,
+ pred_deps->pending_read_mems,
+ &succ_deps->pending_read_insns,
+ &succ_deps->pending_read_mems);
+ concat_insn_mem_list (pred_deps->pending_write_insns,
+ pred_deps->pending_write_mems,
+ &succ_deps->pending_write_insns,
OSDN Git Service
